Sundeep Teki
  • Home
    • About Me
  • AI
    • Consulting
    • Hiring
    • Speaking
    • Papers
    • Testimonials
    • Content
    • Course
    • Neuroscience >
      • Speech
      • Time
      • Memory
  • Coaching
    • Advice
    • Testimonials
  • Training
    • Testimonials
  • Blog
  • Contact
    • News
    • Media

Quality vs. Cost of Large Language Models

16/10/2024

Comments

 
Picture
This image illustrates a significant trend in OpenAI's innovative work on large language models: the simultaneous reduction in costs and improvement in quality over time. This trend is crucial for AI product and business leaders to understand as it impacts strategic decision-making and competitive positioning. Key Insights:
​
  • Cost Efficiency: The cost per million tokens has decreased dramatically by ~10x from ~$36 in March 2023 to about ~$3.5 by August 2024. This suggests technological advancements and increased efficiency in AI model training and deployment, making AI solutions more accessible and scalable.
 
  • Quality Enhancement: The HumanEval scores, which measure coding benchmark quality, have improved from around 67% to over 92% during the same period, representing an improvement of ~33%. The benchmark consists of 164 hand-crafted programming challenges, each including a function signature, docstring, body, and several unit tests, averaging 7.7 tests per problem. These challenges assess a model's understanding of language, algorithms, and simple mathematics, and are comparable to simple software interview questions. This indicates that AI models are not only becoming cheaper but also more capable and reliable.
 
  • Strategic Implications: For businesses, this dual trend of decreasing costs and increasing quality means that AI can be integrated into more applications with better performance outcomes. It allows companies to innovate more rapidly and offer enhanced products or services at lower costs, potentially leading to increased market share.
 
  • Competitive Advantage: Organizations that leverage these advancements can gain a significant edge by delivering superior value to customers. The ability to provide high-quality AI-driven solutions at reduced costs can differentiate a company in a crowded market.

Generative AI startups can capitalize on the trend of decreasing costs and improving quality to drive significant value for their customers. Here are some strategic approaches

1. Cost-Effective Solutions:
  • Affordable Access: By leveraging reduced operational costs, startups can offer competitive pricing, making advanced AI solutions accessible to a broader range of businesses.
 
  • Scalability: Lower costs enable startups to scale their operations more efficiently, allowing them to serve larger markets or expand into new ones without prohibitive expenses.

2. Enhanced Product Offerings:
  • Quality Improvement: With improved quality scores, startups can deliver more reliable and effective AI models, enhancing customer satisfaction and trust.
 
  • Innovation: The ability to offer high-quality outputs at lower costs allows startups to innovate and experiment with new applications, potentially leading to unique product offerings that differentiate them in the market.

3. Strategic Investment in R&D:
  • Focus on Customization: Startups can invest in developing tailored solutions that meet specific customer needs, using generative AI's capabilities for personalization and customization
 
  • Continuous Improvement: By reinvesting savings from reduced costs into research and development, startups can maintain a competitive edge through continuous product enhancements.

4. Operational Efficiency:
  • Automation and Optimization: Generative AI can automate routine tasks, optimizing business processes and freeing up resources for higher-value activities
 
  • Resource Allocation: Efficient cost management allows startups to allocate resources strategically, focusing on areas that maximize impact and profitability

By strategically leveraging these advantages, generative AI startups can enhance their value proposition, attract more customers, and establish a strong foothold in the rapidly evolving AI landscape. Overall, these strategies enable startups to deliver high-quality, innovative solutions at lower costs, providing substantial value to their customers while securing a competitive edge in the market.
Comments

Agentic AI

9/10/2024

Comments

 
What is Agentic AI?
Agentic AI refers to a type of artificial intelligence that can independently act and make decisions in dynamic environments. AI agents go beyond traditional data processing on a wide variety of tasks that mimic humanlike intelligence. It encompasses both hardware agents, such as robots and autonomous vehicles, and software agents, such as conversational agents and chatbots.

Agentic AI has recently regained prominence with the advent of [large language models](https://www.nvidia.com/en-us/glossary/data-science/large-language-models/), such as OpenAI's [GPT-4](https://openai.com/research/gpt-4) and Meta's [Llama 2](https://ai.meta.com/llama/). These powerful models are trained on massive amounts of data and can understand goals, automate work and complete complex tasks. Additionally, conversational agents, such as [ChatGPT](https://chat.openai.com/auth/login) and [Claude](https://claude.ai), which were developed using these large language models, can act on users' goals, gather relevant information from the internet (or relevant knowledge bases) and offer solutions in an iterative fashion.
In this article, you'll learn more about agentic AI, its capabilities and use cases, and its future prospects and possibilities.

Why You Need Agentic AI?
Agentic AI is built on a few fundamental characteristics that include perceiving the environment, making decisions based on perceived data and continuously learning in order to continually improve.
For instance, hardware-based agents are embedded with sensors for vision, sound, motion, heat and humidity. Through these sensors, agents can collect data and learn to make sense of their environment. A popular example is [Mars exploration rovers](https://www.enterpriseai.news/2021/04/01/how-nasa-is-using-ai-to-develop-the-next-generation-of-self-driving-planetary-rovers/) that operate autonomously and perceive their environment to achieve objectives, such as collecting soil and rock samples.
To make decisions in these environments, agents process the perceptual information using intelligent models to select the most optimal course of action. This decision-making mirrors how the human brain processes information, starting from basic perception and building up to higher-level cognitive functions.
Moreover, it's not enough for agents to just perceive and act if they don't continuously improve their intelligence capabilities. This self-learning ability powered by mechanisms such as [reinforcement learning](https://en.wikipedia.org/wiki/Reinforcement_learning) is an essential feature of agents.

Use Cases for Agentic AI
The following are a few scenarios where agentic AI really shines:

Autonomous Vehicles
Self-driving cars or [autonomous vehicles](https://www.embedded.com/the-role-of-artificial-intelligence-in-autonomous-vehicles) are already in production. Companies like Tesla, Uber and Waymo have pioneered their development and deployed the vehicles in real-world environments. Armed with multiple AI modules that operate on various sensors, including [LiDAR](https://en.wikipedia.org/wiki/Lidar), which helps vehicles sense and understand their surroundings, autonomous vehicles represent sophisticated AI agents that can perform complex navigational tasks without human intervention.

Quality Control in Manufacturing

AI agents also find multiple applications in traditional industries, like manufacturing, logistics and agriculture. For instance, modern warehouses are powered by AI agents that can sort, pick up and place items without any human supervision. In agriculture, automated fruit-picking robots can detect fruits that are ready to be picked and harvest them without any damage. Similarly, at different points in the supply chain, robots are used to identify defects and weed out poor-quality products.

Contextual Conversational Agents
ChatGPT has become one of the most popular conversational agents ever, gaining more than a million users in the first week of its launch. These agents can encode a very large context or sequences of words, providing personalized responses to users' queries and demonstrating human-level capabilities across several tasks and domains.
[Autonomous agents, like Auto-GPT and BabyAGI](https://www.fastcompany.com/90880294/auto-gpt-and-babyagi-how-autonomous-agents-are-bringing-generative-ai-to-the-masses), represent another wave of AI agents inspired by large language models like GPT-4 that can automatically complete multiple tasks on their own.

Creative Content Generation
Large language models are typically based on the [transformer model architecture](https://arxiv.org/abs/1706.03762) and are usually trained to predict the next word or sequence of words using a [masked language model training objective](https://huggingface.co/docs/transformers/main/tasks/masked_language_modeling). Given a specific input or prompt, models can generate multiple sentences and paragraphs of text that are consistent with the user input. This has significantly impacted the field of writing and [creative content generation](https://zapier.com/blog/ai-content-creation).

How Agentic AI Pushes the Limits of AI
Conventional AI is rooted in domain-specific applications with limited ability to transfer their learnings from one task or domain to another. AI models that are adept at recognizing objects in images are not as capable in other domains like speech or language. To achieve true humanlike intelligence, AI needs to be adaptable across several tasks and domains.
AI agents are a promising step in this direction as they combine perception, decision-making and self-learning abilities in dynamic environments (*eg* autonomous vehicles driving in the real world). While conventional AI may power some of the underlying capabilities, such as image segmentation and object detection, these capabilities cannot help the vehicle drive safely in real-world environments. This is where agentic AI shines.


Limitations of Conventional AI
Unlike agents that continually learn from interactions with their environment, conventional AI remains static. Conventional AI also struggles to operate effectively across various domains.
In the following sections, you'll take a deeper look at some of the shortfalls of conventional AI and how agentic AI overcomes these challenges.


Limited Learning Capacity
Conventional AI models learn from data, and the knowledge that they learn during model training is confined to the specific data set and the domain. While some models are capable of doing well on related tasks with [transfer learning](https://en.wikipedia.org/wiki/Transfer_learning), the transferred knowledge is usually restricted to the same domain. For instance, models that are trained to identify sentiment in text are not capable of detecting sentiment in voices. This limited learning capacity of conventional AI models is a bottleneck in the development of humanlike intelligence.
Meanwhile, agentic AI is endowed with self-learning capacity. Agents actively interact with their environment and constantly update their knowledge and capabilities.


Narrow Task Focus
Related to their limited learning capacity, another shortcoming of conventional AI is the narrow task or domain in which they excel. While some AI models are accurate when it comes to specific tasks and domains, they tend to significantly underperform when the nature of the data or the task changes.
In comparison, AI agents can learn new tasks as needed. This makes them more versatile and adaptable than conventional AI models, which operate well in narrow task domains.

Difficulty Handling Ambiguity
Conventional AI also struggles to operate in ambiguous situations. For instance, a speech recognition model that is trained in American English may struggle to recognize British English accents. The difference in distribution between the data in the training set and the real-world test set evolves over time, and this results in the models being unable to handle novel and ambiguous inputs.
AI agents are adept at handling ambiguity as they constantly sense their environment for changes and learn to respond to these changes. This feedback-based learning helps them adapt to novel or ambiguous features in their environment and be more robust and reliable than conventional AI models.

High Maintenance and Rigidity
Another downside of conventional AI is its high requirements for maintenance and the rigidity of its performance.
Conventional AI models need to be constantly monitored, updated and retrained to better reflect the real-world data, or they'll struggle from [data drift](https://www.openlayer.com/blog/post/surefire-ways-to-identify-data-drift) and poor performance. The models can also be rigid in the sense that they can perform very well only in highly specific conditions and may struggle with changing environments.
AI agents do not have substantial requirements for high levels of maintenance and monitoring as they're capable of monitoring their own responses to adapt to their environment. As long as the goals of the agents remain the same, agents do not need to be constantly updated, unlike conventional AI models, which require periodic retraining.


Future Prospects and Possibilities of Agentic AI
Agentic AI has a [bright future](https://ts2.space/en/the-rise-of-ai-agents-the-future-of-artificial-intelligence) that's bolstered by the breakneck growth and development in the field of large language models and [generative AI](https://www.mckinsey.com/featured-insights/mckinsey-explainers/whats-the-future-of-generative-ai-an-early-view-in-15-charts).
Software-based agents built on large language models have boosted the efficacy of conversational intelligence, knowledge search and management, content generation and other business applications.
Hardware-based agents, such as robots, also stand to benefit from agentic AI. Agentic AI can empower several fundamental aspects of robotics, such as design, generation of realistic simulations of robotic systems as well as human-robot interaction. According to [Gartner](https://www.gartner.com/en/topics/generative-ai), up to 40 percent of enterprise applications will have embedded conversational AI by 2024, and up to 100 million workers will engage agents to assist in their work by 2026.


Conclusion
Artificial intelligence is now ubiquitous, powering multiple consumer and enterprise applications across industries. Agentic AI represents a type of AI that can act and make decisions to accomplish specific goals in real-world environments. While robots as a form of agentic AI have been around for a while, recent advances in conventional AI based on large vision and language models are paving the way for a new breed of AI agents.
In this article, you learned about the elements of agentic AI, how it's different from conventional AI, what its real-world applications are as well as what the future prospects and potential of agentic AI are. In particular, conversational agents like ChatGPT are increasingly being adopted for various use cases. These chat agents leverage a different style of technology stack that includes new elements, such as [vector databases](https://www.singlestore.com/built-in-vector-database/) that are used for storing smaller or compressed pieces of information and retrieving relevant information chunks based on a [user query in chat applications].
Comments

    Archives

    July 2025
    June 2025
    May 2025
    April 2025
    March 2025
    February 2025
    January 2025
    October 2024
    September 2024
    March 2024
    February 2024
    April 2023
    December 2022
    November 2022
    October 2022
    September 2022
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    March 2022
    February 2022
    December 2021
    October 2021
    August 2021
    May 2021
    April 2021
    March 2021

    Categories

    All
    Ai
    Data
    Education
    Genai
    India
    Jobs
    Leadership
    NLP
    RemoteWork
    Science
    Speech
    Strategy
    Web3

    RSS Feed


    Copyright © 2025, Sundeep Teki
    All rights reserved. No part of these articles may be reproduced, distributed, or transmitted in any form or by any means, including  electronic or mechanical methods, without the prior written permission of the author. 
    Disclaimer
    This is a personal blog. Any views or opinions represented in this blog are personal and belong solely to the blog owner and do not represent those of people, institutions or organizations that the owner may or may not be associated with in professional or personal capacity, unless explicitly stated.
                                                                                                                                                                                 [email protected] 
​​  ​© 2025 | Sundeep Teki
  • Home
    • About Me
  • AI
    • Consulting
    • Hiring
    • Speaking
    • Papers
    • Testimonials
    • Content
    • Course
    • Neuroscience >
      • Speech
      • Time
      • Memory
  • Coaching
    • Advice
    • Testimonials
  • Training
    • Testimonials
  • Blog
  • Contact
    • News
    • Media