What is Natural Language Processing?

What Is Natural Language Processing?

What is natural language processing? Simply put, it’s a field of computer science and Artificial Intelligence (AI) that allows computers to understand human language and process it in a way that can be manipulated and worked on. NLP is usually split in two parts, natural language understanding (NLU) and natural language generation (NLG). In software development, NLU is usually the aspect that is harder to do, therefore it is more time consuming. Hence, the performance is key for all NLU tasks.

Steps of NLU

When working with natural language processing there are three main steps that need to be completed.

1. Lexical Analysis – at this step the software takes the inputted text and divides it into smaller chunks, such as sentences and words. This is the preliminary step to make sure the text is ready to be inspected one sentence at a time.
2. Syntactic Analysis – at this step the software takes the lexically analyzed text and starts looking at the sentence structure to gather information regarding the words and their relationships with each other. This is when un parse-able sentences (if any) get thrown out.
3. Semantic Analysis - In this step, each word is defined and mapped directly from the dictionary to validate the meaningfulness of the sentence.

Tasks of NLU

• Part-of-speech tagging (POS), maps each word to its relative syntactic role: verb, noun, adjective, adverb, etc.
• Chunking (CHK), tags a certain word/segment inside a paragraph to its relative syntactic role.
• Named-entity Recognition (NER), tags a certain word inside a paragraph as an identifier of meaning. That is, location, date, person, etc.
• Semantic-role labeling (SRL), sometimes called shallow semantic representation, uses a top down parsing approach (in most cases) and looks at the relationships between the words within sentence to understand the meaning of the word. (see “understanding SRL” for more info)
• Syntactic Parsing (PSG), processes the sentences and assigns syntactic structure to it. (see “understanding PSG” for more info)

POS, CHK and NER are fairly fast processes, while SRL and PSG are much more time consuming and involved. In most cases, one may expect the following performance stats in relationship to each other:

   CHK - 1x
   NER - 1x 
   POS - 1.5x
   SRL - 36x
   PSG - 74x

Understanding SRL

As we said earlier, SRL looks at every sentence and the words within to understand the relationship between each other and map each word to its role. Let’s look at the following sentence for example:

"Company wrote a software with most modern technologies"

If we were to convert this sentence into a mathematical expression we will get something like this:

∃ (x, y, z)     Writing(x) ∧ Writer(x, Company) ∧ 
                WrittenObject(x, y) ∧ Software(y) ∧ 
                WrittenWith(y, z) ∧ Technology(z)

In this example we can tell that “Company”, which is the agent did some action to software, (called result) with some instrument (“most modern technologies”). These labels are called semantic role labels (or shallow semantic representations). Other common roles include:

• Agent – person or object that is performing some action.
• Experiencer – person or object that is experiencing some action.
• Theme – theme or item that is getting actioned on.
• Result - the output of the action that was performed.
• Instrument – the object/item used to complete the action.
• Source – the item/place/object that usually is preceded with “from” tag.
• Goal – the item/place/object that usually is preceded with “to” tag.

Understanding PSG

PSG, or Syntactic Parser, as we said processes sentence and assigns the structure to it. So let’s look at the same example above to further understand what is meant by this:

"Company wrote a software with most modern technologies"

Step 1: First, parse the tree out based on words:

• “Company” – Noun Phrase (NP)
• “Wrote” – Verb Phrase (VP)
  • “a” – determiner (Det)
  • “software” – noun phrase (NP)
    • “with most modern technologies” – prepositional phrase

Step 2: Assign a top down tree to the sentence:

Ambiguity in SRL

The biggest problem when constructing the SRL is the ambiguity similar to constructing a compiler parsed tree. The tree we designed above can be redrawn a in such a way that the meaning becomes:

"Company wrote a software, which had the most modern technologies"

Compared to our initial sentence of:

"Company wrote a software with the most modern technologies"

Currently, there are algorithms that deal with ambiguities like CKY parsing (Dynamic Programming) or using Chomsky Normal Form. This process is called syntactic disambiguation.