AI for All, Great AI for Some - NIQ
Commentary

AI for All, Great AI for Some

Quality Predictive Modeling and Analysis Demand Best-in-Class Data

Commentary

AI for All, Great AI for Some

Quality Predictive Modeling and Analysis Demand Best-in-Class Data


Every day, NIQ empowers retail and manufacturing clients across 90+ countries, 1.4M+ stores, and trillions of records with artificial intelligence (AI) and machine learning (ML) to drive actionable insights.

NIQ is the premier partner to today’s top manufacturers and retailers. Here’s why:

  • The highest quality raw data ​
  • Common key data sets​
  • Trustworthy analysis and modeling ​
  • Quality control supported by human intelligence​

NIQ is the premier partner to today’s top manufacturers and retailers

You’ll have to forgive our skepticism when, seemingly overnight, virtually every organization began to position itself as an “AI organization.” ​

Where there is no doubt: Brands in nearly every industry have both an increased awareness of AI and an increased focus on being AI-forward. Manufacturers (and the organizations that supply them with data) are no exception. ​

Retailers are also taking the leap. A recent study from Honeywell reports 97% of retailers surveyed claim to be using AI technologies – or are in active discussions to use them – for select use cases or regions, on a large scale, or in a pilot phase1.  ​

But it takes a lot more than just calling yourself an AI-driven organization to become one. The sheer amount of data demanded to drive truly meaningful outcomes for both brands (and the data suppliers they lean on) is enough to potentially overwhelm even the most seasoned team of data scientists. That massive data volume combined with poor data quality, noisy signals, and inaccuracies in analysis and modeling is a recipe for strategic misfires.  ​

Unless, of course, you work with the right partner, one with a proven track record of leveraging AI to analyze and model data at scale – a partner like NIQ. ​

The right data supplier should offer you the “full package.” They should allow you to license the high-quality raw data you need to train your models to deliver accurate, actionable insights. They should help you merge data sets strategically so that you can benefit from cross-sectioning and slashing the data every which way. And they should generously share hard-won experience in analysis, modeling, and quality control to make sure you’re getting as much of the benefit as possible with the least amount of risk necessary. Phrased differently: They should be doing much of the heavy data lift, and you should be benefiting from the years of experience they gained by truly being an AI organization before … well … everyone else was an “AI organization.”  ​

The plain truth is that every retail and manufacturing organization must be – at least to a certain extent – an AI organization now, or risk falling behind.  ​

Do you trust your data inputs enough to stake the future of your business on the outputs they generate?​

Fast Facts

NIQ Artificial Intelligence and Machine Learning Data by Volume

163B

… total records processed (approximately equivalent to total transactions processed by Mastercard)2

125M

… items coded on our proprietary global data system (10x the total number of items cataloged by Amazon)3 

60K

… reports delivered monthly (6x the total number of reports published by World Bank and World Economic Forum annually)4

Here’s why NIQ is the premier partner to today’s top manufacturers and retailers:​

  • The highest quality raw data ​
  • Common key data sets​
  • Trustworthy analysis and modeling ​
  • Quality control supported by human intelligence​

High-quality raw data​

AI doesn’t differentiate between good and bad input data. This renders the stakes exceptionally high when you consider that the AI models themselves are trained using between one and (potentially) infinite data sets. The more accurate the data an organization works with, the more accurate the model prediction will be. The opposite is also true: The more “bad” data that an organization feeds into models, the more those models will generate errors in prediction – errors that can increase exponentially in number as more bad data is absorbed. ​

As a leading industry provider of relevant data sources, NIQ provides its clients with professionally run and monitored panels that offer both quality and coverage. The result? Our clients work from a data repository, or data lake, that is more accurate than any other. For example, many brands claim a 50–60% level of data accuracy. NIQ, by comparison, delivers 90% accuracy. Anything less accurate simply isn’t sufficient for superior outcomes, and any organization leveraging inferior data sets is at a competitive risk. 

How NIQ Supports Retailers and Manufacturers with High-Quality, Raw Data ​

Retail Measurement Services (RMS) provides data that details “what, when, and where” products were sold. NIQ offers the most robust data sets in the world when it comes to product granularity, item-level data, etc. ​

Consumer Panel Services (CPS) measures purchases intended for use at home and provides household demographics and consumer behavior behind those purchases (where they shop, who they are, etc.). NIQ offers data that goes beyond FMCG panels to offer a truly unique depth and breadth of data collection – such as retail data from retailers who do not participate in RMS, consumer app purchase behavior, and causal relationships. ​

Here’s a simplified equation to help think about NIQ’s data sets:

RMS – “the buy”​

​+

CPS ​ – “the why behind the buy”

=

the most complete picture of sales among all buyers in the industry.​


What does this all mean for your organization’s AI efforts? The combination of RMS and CPS allows our customers to leverage the most robust and granular reference data set available for both modeling and analysis. Period.​

Common key data sets​

Procuring clean, raw data is just the start. Data sets are only beneficial if they can use a common language to “talk” to each other. How exactly does data communicate? Through common “key” data sets, like EANs or UPCs. Think of these common data sets as the glue that binds tens, hundreds, thousands, or even millions of previously disparate pieces of data together. As soon as data sets can talk, they can start to deliver insights and modeling.

Once a common key data set in in place, the real magic can happen through a process called cross-coding, which enables organizations to build new models that surface new insights that pave the way for unparalleled leaps in analysis – all, notably, with minimal human intervention and at a pace that was previously unfathomable.​

Enter NIQ. Not only do we offer the highest quality, cleanest data sets in the industry, our data also serves as the aforementioned glue between previously unconnected data sets. By marrying data from NIQ to internal data from manufacturers and retailers, our clients get the full view of what’s happening now alongside a predictive model to anticipate what’s going to happen in the future.

These insights propel both retailers and manufacturers beyond the illusion that data only ever reveals a singular truth about a given scenario, enabling them to successfully slice the data in myriad ways based on their specific use cases for the data sets. Instead of walking away with one dry prediction for what to do, retailers and manufacturers gain nuanced, flexible insights that reveal multiple possible realities. ​

Imagine what your organization could do if it were able to marry 100+ data sets together – in near real time – to drive analysis and forecasting.​

Fast Facts

NIQ Artificial Intelligence and Machine Learning Data by Volume

377M

… items cross-referenced every year across 77 countries

2.4M

records accurately categorized each week via cross-coding

An average of

1.5M

unique characteristics in support of new items created every single hour

Global data collected for

6.9M

… items every single day

Trustworthy analysis and modeling​

It’s not just flawed, incomplete, or poor-quality data that results in faulty output from artificial intelligence and machine learning efforts. The risk that retailers and manufacturers face when it comes to the quality of these systems’ output extends to other areas as well. They must remain vigilant to ensure the analysis and modeling that takes place across their data is accurate, trustworthy, and optimized for efficiency and performance.

In order to minimize risk and avoid common pitfalls, NIQ maintains a data-centric mindset when it comes to building and applying tech-enabled data science and modeling, even when keeping diverse, unique cases in mind. This is made possible thanks to specialized talent dedicated to identifying and coding AI/ML data projects that are customized and tailored to exceed our clients’ expectations.

Fast Facts

NIQ Brings the Receipts When It Comes to Talent

34

PhDs on our dedicated AI/ML team

Over

100

talks delivered in the AI space

15+

papers published at top AI conferences

33

AI-related patents since 2015

Quality control supported by human intelligence​

The standard data science project includes the stages of scoping, collecting data, training the model, and deploying the model in production. Nascent AI partners stop there, relying on the data and the modeling to be accurate. This is problematic for two reasons. First, it ignores the biggest risk currently associated with generative AI: data hallucinations. This phenomenon is the result of AI models’ occasional tendency to identify otherwise imperceptible patterns in data and to incorporate those patterns into their outputs, which can lead to inaccurate conclusions. Second, it can lead to AI algorithms that fail to account for subtle, but critical, nuances between data sets. Any time data is brought together from different sources, there is the risk of delivering misleading outputs – outputs that organizations nevertheless stake their future planning on.​

These are not risks we recommend taking, which is why NIQ takes a different approach – one that applies human intelligence to the quality control protocols for modeling and analysis. With expert humans in the loop at strategic junctures, it’s possible to mitigate those risks and greatly decrease any of their potential negative impacts. ​

What does this look like in practice? NIQ brings in Machine Learning Operations (MLOps) specialists who develop their own methods and algorithms to audit the quality of the AI model’s output, essentially finding ways to separate the “good” predictions from the “bad” ones. They also ensure the model is working optimally amid changing demands by monitoring algorithms’ metrics, improving performance, and establishing best practices – ultimately resulting in more systematic, repeatable, and well-maintained ML and AI projects.

For example, as an organization adds different data from new points of sale or customer interfaces, or as the business environment grows, MLOps specialists evaluate the results and take a proactive approach to the information as it is reported on. When those using the analytics query the results or notice any misbehavior, MLOps specialists identify the source of the problem and remedy it. This becomes more important as organizations ingest larger volumes of data to fuel business intelligence insights. And this is why NIQ has moved from being a traditional market research company to being a trusted provider of prescriptive data analytics, powered by innovative technology.

What’s possible for brands and retailers – current and future-state business use cases​

When these foundational elements of AI and ML are in place, retailers and manufacturers set themselves up to truly transform their strategies and shape their own destiny. We use the word “strategies” (plural) intentionally, as the benefits of AI/ML are not limited to any one area. AI/ML can have surprisingly universal applications, from customer operations to marketing and sales to product innovation.​

The universal application of AI is partly a result of the extensiveness of the data and the way sets can be linked, and partly a result of the democratization of AI. Consumer-facing tools like ChatGPT have encouraged adoption and familiarity with individuals at every level, which means non-technical, non-data scientists are able to leverage AI. These individuals can ask and answer questions without having to involve a data scientist, which allows them to move quickly and flexibly as they ponder the possibilities and make decisions. ​

The resulting cross-functional collaboration and information sharing is driving continuous improvements and innovations and making analysis far more accessible than ever before. ​

Think of generative AI applied to CPG, tech & durables, and retail in a similar way. Imagine typing a question into a platform provided by your data supplier (much like you may currently be asking ChatGPT) and getting accurate responses based on real-time industry data. An example: “What consumer ingredient trends am I not aware of that I should be monitoring?” Sound too good to be true? Well, it’s coming – and fast.​

Similarly, imagine a world where a CMO is able to query – sans analyst – relative to the success of a marketing campaign: “Did we reach the right consumers? What consumer insights should we glean from this campaign? Should we invest more dollars in this effort?” Also coming soon to a screen near you.​

While the future is exciting to think about, there is no shortage of impressive present-day AI/ML applications. NIQ’s unique, AI/ML-enriched data sets and tools are already delivering game-changing results to it retailer and manufacturer partners, including: ​

  • Operational efficiency. Use cases like item coding automation, automatic matching, and sustainable quality control are powered by AI/ML technologies and have significantly improved operational efficiency.
  • Enhanced data quality and insights. Data Validations, BASES Architect, and Media Measurement – solutions built on and powered by AI/ML – allow retailers and manufacturers to identify patterns, trends, and relationships in large data sets, enabling businesses to make data-driven decisions and stay ahead in the competitive market.​
  • Improved product performance and market forecasting. BASES Product AIgfknewron Forecasting, and gfknewron Price Elasticity are AI/ML applications that have helped manufacturing clients optimize product performance, measure market responsiveness, and generate accurate forecasts, leading to better decision-making, strategic planning, and overall business growth.

Our framework helps clients create value with GenAI 

Advisory


Help partners understand how this technology will impact consumers, retailers, manufacturers and market research​

  • Assessing the landscape of GenAI to continually inform optimal strategic applications​
  • Articulate a point of view on the transformative impact of GenAI on the consumer value chain​
  • Identifying opportunities for growth and innovation​

Enablement


What data and processes clients need in order to maximize value of first and third-party solutions​

  • Providing data and frameworks for supported use cases​
  • Identifying and integrating relevant data sources​
  • Implementing processes for data quality and security​
  • Facilitating access to cutting-edge tools and platforms​

Testing


What testing must ​ be done to manage the risks and limitations​

  • Establishing rigorous testing protocols for GenAI models​
  • Identifying and mitigating potential risks and biases​
  • Ensuring transparency and explainability of AI systems​
  • Performance evaluation and benchmarking​
  • Adhering to industry standards and best practices​

Solutions


What are the sophisticated, high potential solutions clients need (which NIQ is developing) ​

  • Designing GenAI powered solutions tailored to client needs​
  • Integrating GenAI into existing systems and workflows​
  • Developing industry-specific use cases and applications​
  • Measuring and demonstrating the value of GenAI to clients​

Conclusion​

If, like most manufacturers and retailers, you have set your sights on realizing the full potential of AI and ML, don’t try to go it alone. NIQ was in the business of AI long before it was trendy, and no other organization in the CPG, technology & durables, or retail space has a more robust, granular, or complete data set. When you work with NIQ, you put a proud, rich history of quality data acquisition to work for your organization and lay the groundwork for an even richer future.​

We look forward to partnering with you and your team.

Footnotes
Footnotes
  1. Navigating the AI Revolution.” Honeywell and OnePoll. May 2023. https://sps.honeywell.com/us/en/campaigns/productivity/discover-frictionless-retail.
  2. “Number of Credit Card Transactions per Second, Day & Year.” Capital One. Last updated June 9, 2023. https://capitaloneshopping.com/research/number-of-credit-card-transactions.​
  3. Dayton, Emily. “Amazon Statistics You Should Know: Opportunities to Make the Most of America’s Top Online Marketplace.” BigCommerce. Accessed November 14, 2023. https://www.bigcommerce.com/blog/amazon-statistics/.​
  4. World Bank Open Data.” The World Bank. Accessed November 14, 2023. https://data.worldbank.org.

Stay ahead by staying in the loop.

Don’t miss the latest insights, offerings, and opportunities from NielsenIQ.

By clicking on sign up, you agree to our privacy statement and terms of use.