journalfront

Ai2 Launches OLMo 2: Competing with Meta's Llama

2024-11-27

There's a significant development in the world of AI as a new model family emerges. Ai2, the renowned nonprofit AI research organization founded by Paul Allen, has released OLMo 2, the second installment in its OLMo series. This open source AI model holds great promise and is set to make waves in the industry.

Unlock the Potential of Open Source AI with OLMo 2

Introduction to OLMo 2

OLMo 2 is a remarkable addition to the AI landscape. It stands out as one of the few models that can be reproduced from scratch. With two models in its family - OLMo 7B with 7 billion parameters and OLMo 13B with 13 billion parameters - it offers different levels of problem-solving capabilities. These parameters roughly correspond to the model's proficiency in handling various tasks.The training of OLMo 2 involved a vast data set of 5 trillion tokens. Tokens represent individual units of raw data, and 1 million tokens is approximately equivalent to 750,000 words. The training set was carefully curated, including websites filtered for high quality, academic papers, Q&A discussion boards, and math workbooks both synthetic and human-generated. This extensive data set has contributed to the model's impressive performance.

Performance and Comparisons

Like most language models, OLMo 2 7B and 13B can handle a wide range of text-based tasks such as answering questions, summarizing documents, and writing code. Ai2 claims that these models are competitive in terms of performance with open models like Meta's Llama 3.1 release. In fact, not only do they show a dramatic improvement in performance across all tasks compared to the earlier OLMo model, but OLMo 2 7B even outperforms Llama 3.1 8B. This makes OLMo 2 the best fully-open language model available to date.The ability to reproduce the model from scratch and the open source nature of its components give researchers and developers the opportunity to explore and innovate. It promotes technical advancements and leads to more ethical models. Additionally, the open access to the data, recipes, and findings allows for verification and reproducibility, reducing the concentration of power and creating more equitable access.

Open Source and Safety

The Open Source Initiative's definition of open source AI was finalized in October, and OLMo 2 meets these criteria. All the tools and data used to develop OLMo 2 are publicly available, enabling the open source community to build upon and contribute to its development.There has been some debate about the safety of open models recently, especially with reports of Llama models being used by Chinese researchers for defense tools. However, Ai2 engineer Dirk Groeneveld believes that the benefits of open models outweigh the harms. He emphasizes that this approach promotes technical advancements and ethical models, and is a prerequisite for verification and reproducibility.In conclusion, OLMo 2 is a game-changer in the field of AI. Its open source nature, impressive performance, and potential for innovation make it a valuable asset for researchers and developers. With its components available for download from Ai2's website under the Apache 2.0 license, it is set to have a significant impact on the future of AI.

As large enterprises strive to integrate AI into their platforms and processes, a significant challenge emerges. Generative AI requires memory and continuous training data updates to be practically useful. This has led to the emergence of 'Live AI', with several startups like Cohere, Writer, and Pathway actively working in this space. Pathway, in particular, has just raised a $10 million Seed round, aiming to build live AI systems that think and learn in real-time like humans.

Investors and Their Impact

The Seed round was led by TQ Ventures, with participation from Kadmos, Innovo, Market One Capital, Id4, and angel investors. An interesting investor in Pathway is Lukasz Kaiser, the co-author of Transformers and a key researcher behind GPT o1 from OpenAI. These investors believe in Pathway's potential and are supporting its growth.

Pathway's Offering

Pathway's offering includes 'infrastructure components' that power live AI systems. By feeding on structured and unstructured data, enterprise AI platforms can make decisions based on up-to-date knowledge. Customers such as NATO and La Poste, the French post office, have already recognized the value of Pathway's solutions.Zuzanna Stamirowska, Co-Founder and CEO of Pathway, emphasizes the importance of dealing with knowledge and memory in deep learning and LLM assistants. Currently, an LLM acts like a smart intern on the first day, unable to truly memorize information. Pathway enables developers to build a pipeline for feeding live data into AI systems during the prompting stage of building LLM or Gen AI applications.

The Team Behind Pathway

Stamirowska has assembled an impressive, highly technical team to achieve the startup's goals. Her co-founders are CSO Adrian Kosowski and CTO Jan Chorowski, who previously worked with the "Godfather of AI", Geoff Hinton. Stamirowska herself is the author of a state-of-the-art forecasting model for a complex network in the maritime trade, published by the Academy of Sciences of the US."The company started with an idea that came to my mind on a sunny morning in Chicago," she said. "Accompanying a friend to a scientific conference in theoretical computer science led to a small disagreement, which inspired me to start my own thing. I took out my laptop and began reaching out to people in my network to move forward. I still remember the taste of that coffee."

Pathway's Competitive Position

When asked about Pathway's position against other startups in the space, Zuzanna Stamirowska pointed out that in GenAI engineering and knowledge management, Cohere and Writer are beside them in the latest Gartner Quadrants. In enterprise deals, they often encounter Palantir for AI transformation tenders, although Palantir is less product-oriented than Pathway.Commenting in a statement, Schuster Tanger, Co-Managing Partner and Co-founder at TQ Ventures, praised Pathway. He said, "Zuzanna and the team at Pathway possess bleeding-edge insights and expertise in one of the most exciting fields in modern business. The response from the developer community has been powerful."

Incoming president Donald Trump is currently giving serious thought to naming an "AI czar" within the White House, as reported by Axios. This move could potentially have a significant impact on the future of artificial intelligence within the government. If such a policy position is appointed, they would be tasked with facilitating the coordination of federal regulations and the utilization of AI by the government. It's important to note that an AI czar would not need Senate confirmation, enabling them to swiftly commence working towards the administration's goals, as pointed out by Axios. The role of an AI czar might potentially be merged with a "crypto czar" role that the White House is also reportedly contemplating. This combined role would complement the agency chief AI officers established under President Joe Biden's AI executive order. However, Trump has the option to eliminate these officers and their respective offices in favor of a more centralized governance approach. At present, no names have been proposed for this crucial position. But it is widely known that Tesla CEO Elon Musk and Vivek Ramaswamy, the co-leaders of the extra-governmental organization known as DOGE, will play a significant role in determining who will hold the AI czar position. Their expertise and influence in the field of technology are likely to shape the direction of this important role.

Significance and Implications

The appointment of an AI czar holds great significance as it could lead to more streamlined and coordinated efforts in the development and implementation of AI policies. It would help ensure that the government is at the forefront of leveraging AI technology for the betterment of society. Without the need for Senate confirmation, the AI czar can quickly get to work on addressing the challenges and opportunities presented by AI. This could potentially result in faster decision-making and the implementation of effective AI strategies. Moreover, the potential combination of the AI czar role with the crypto czar role adds another layer of complexity and potential synergy. It shows that the White House is taking a comprehensive approach to addressing the emerging technologies and their implications.

Elon Musk's Potential Impact

Tesla CEO Elon Musk is a renowned figure in the tech industry and has extensive experience in the field of AI. His insights and leadership could bring a unique perspective to the role of AI czar. Musk has been at the forefront of developing and promoting AI technologies through Tesla's autonomous driving projects. His understanding of the technical aspects and potential risks associated with AI makes him a valuable candidate for this position. His influence could help shape the government's approach to AI, ensuring that it is developed in a safe and responsible manner.

Vivek Ramaswamy's Role

Vivek Ramaswamy, as the co-leader of DOGE, brings a different set of skills and perspectives to the table. DOGE is an extra-governmental organization that has been actively involved in the blockchain and cryptocurrency space. Ramaswamy's expertise in this area could potentially contribute to the integration of AI and blockchain technologies. His understanding of the decentralized nature of these technologies could help the government navigate the complex landscape of emerging technologies and find innovative solutions. His input could also help ensure that the government's AI policies are in line with the evolving needs of the digital economy.