‘You shall know a word by the company it keeps’ – John Firth, 1957
NLP enrichment goes beyond simply tracking or counting the number of keywords in a data-set: it statistically interrogates the ‘company’ a word keeps (ie: other related words or phrases within the same tweet), and then the company that connected words keep, and so on and so forth. This is repeated millions of times to analyse and map connections into a single, coherent model, transforming unstructured data into meaningful objects and relationships – people, places, products, ingredients, occasions, events – and finding the correlations between them.
NLP in practice
When executed using large data-sets, the results can be extremely powerful. For example, we used the Nest and NLP analysis to combine social media and environmental weather data to find the conversation signals and physical triggers that would pre-empt a cold and flu virus flare-up. Our results predicted an outbreak with hyper-local accuracy a full four days ahead of any local authority advice.
Find out more