Unlock the Potential of Real-time Video-based Conversational AI
Initial Focus on Vertical Markets
eSelf's initial focus is on various verticals including education, sales, financial services, real estate, and health and wellness. These verticals present unique opportunities for the development and application of real-time video-based conversational AI agents. By targeting these specific industries, eSelf aims to provide tailored solutions that meet the specific needs of each sector.For example, in the education sector, these agents can assist students with their studies, provide answers to questions, and offer personalized learning experiences. In sales, they can help sales representatives engage with potential customers, provide product information, and schedule appointments. In financial services, they can offer financial advice, help with account management, and provide real-time market information. In real estate, they can assist with property searches, provide property details, and schedule viewings. And in health and wellness, they can offer health advice, provide information about medical conditions, and schedule appointments with healthcare providers.
Faster Query/Interaction Response Time
One of eSelf's early pitches is that its agents' current query/interaction response time is under 1.7 seconds. This is claimed to be faster than voice responses from other leading companies like OpenAI, making the interactions more lifelike. The startup has launched a "no-code" creation studio that allows its customers to build their own agents. These agents can operate common apps such as Calendly, Salesforce, and others to schedule appointments, access collateral content, and reference data just as a human assistant might do.For instance, when a customer schedules an appointment using an eSelf agent, the agent can seamlessly integrate with Calendly to find available time slots and schedule the appointment. Similarly, when a customer needs access to collateral content, the agent can quickly access and provide the relevant information from Salesforce or other apps. This seamless integration and fast response time enhance the user experience and make the interactions more efficient.
Powering Millions of Real-time Conversations
eSelf claims to have powered "millions" of real-time conversations in stealth mode. Current customers include Christie's real estate and Brazilian bank AGI. These successful deployments demonstrate the potential and effectiveness of eSelf's real-time video-based conversational AI agents.For example, in Christie's real estate, the eSelf agent can assist potential buyers with property searches, provide detailed property information, and schedule viewings. In Brazilian bank AGI, the agent can help customers with account management, provide financial advice, and handle various banking transactions. These real-world examples showcase the practical applications and value of eSelf's technology.
Unlikely Origin Story
eSelf is co-founded by Dr Alan Bekker (CEO) and Eylon Shoshan (CTO), and it has an interesting origin story. Bekker, who lives in Tel Aviv, originally wanted to become a rabbi. However, after approaching the chief rabbi in his school and not getting satisfactory answers to his big questions about God and faith, he decided to drop out of rabbinical school and pursue a career in engineering, physics, and computer science.He had a longstanding interest in math and signed up for a course in machine learning by mistake. This course gave him a glimpse into another kind of omniscience, where he realized that with mathematics, one can understand natural language, vision, and speech. This discovery sparked his passion for understanding humans and the world.
First Startup and Acquisition by Snap
Bekker's first startup was Voca, which focused on building AI voice assistants for call centers. It attracted large customers like American Express and TNT (now a part of FedEx). Eventually, Snap acquired Voca in 2020 for $100 million, although the details of how the tech would be used were unclear.Bekker then took on a role as the head of Conversational AI at Snap and led a team in building conversational AI tech. However, it took almost three years and the arrival of generative AI for this technology to find a landing point in a product. By the time Snapchat launched My AI in 2023, Bekker had already left the company.
Co-founding eSelf and Targeting the "Agentic" Opportunity
Meanwhile, Bekker teamed up with Eylon Shoshan to co-found eSelf. In a world where the tech industry is becoming crowded with companies targeting the "agentic" opportunity, eSelf aims to stand out by offering more interactive and humanlike products using video and voice in the response.By tapping into advances in Generative AI, eSelf is creating building blocks to craft agents that can carry out various tasks. Startups and established players like Salesforce are also coming out with agents for specific use cases. It's early days, but it will be interesting to see which of these startups land investors and gain traction with paying users.
Model Building and Training
One of the significant challenges in the development of real-time video-based conversational AI agents is the building and training of models. Bekker said that eSelf has not built its own LLMs from scratch but has fine-tuned Meta-created Llama models. This approach allows customers who already have chatbots built on platforms like GPT to switch to using eSelf's technology with relatively lower latency.By fine-tuning existing models, eSelf can leverage the advancements in machine learning and natural language processing while reducing the time and resources required for model building. This approach also enables eSelf to offer a more cost-effective solution for customers looking to build and operate real-time video-based conversational AI agents.
Stay Informed with TechCrunch's AI Newsletter
News
In the world of AI, there are always exciting developments happening. This week, we bring you a roundup of the most notable news:
Image Credits: Kirillm / Getty Images
A test for AGI is getting close to being solved, but it points to flaws in the test's design rather than a breakthrough. Amazon is establishing a new R&D lab in San Francisco focused on building "foundational" capabilities for AI agents. OpenAI's video generator, Sora, is now available to most ChatGPT Pro and Plus subscribers, while those in Europe are out of luck. China has opened an antitrust probe into Nvidia's acquisition of Mellanox. Yelp has released AI-powered review insights to analyze sentiment and highlight categories. Google has signed a deal to spin up carbon-free power for data centers. Reddit has debuted Reddit Answers, a conversational AI feature. X (formerly Twitter) has a new image generator, Aurora, tuned for photorealistic rendering.
Research paper of the week:
Image Credits: Bryce Durbin / TechCrunch
A team from Ai2 and UC San Diego has created an AI model that can predict 100 years of climate patterns in 25 hours. Called Spherical Dyffusion, it starts with basic climate science knowledge and applies transformations. It can run on modest hardware and has limitations, but the researchers plan to refine it. Ai2 also released the second generation of its climate-modeling AI, Climate Emulator.
Model of the week
Sora is getting attention, but a new video-generating model from MIT CSAIL and Adobe Research is potentially more exciting. Called CausVid, it can start playing videos as it generates them, providing a preview. The researchers plan to release an open source implementation soon.
Grab bag:
The group that leaked access to Sora last November has published essays explaining their actions. They wanted to denounce the exploitation of creatives for R&D and PR. As they wrote in a post, "We called on artists to think beyond proprietary systems and the limitations of prompting a model mediated by big tech."
As always, we're open to your feedback. Drop us a line with your thoughts anytime.Addressing the Talent Shortage
Finding good site reliability engineers is a daunting task, as there is a high churn rate in the field. The modern IT stack continues to grow more complex, making it nearly impossible for humans to keep up. As NeuBird's CEO, Rao, explained to TechCrunch, "It's very hard to find good site reliability engineers. There's a lot of churn. It doesn't help that the modern IT stack just keeps getting more and more complex. Humans alone cannot possibly keep up with this kind of change."The Hawkeye Solution
To tackle this increased complexity, NeuBird developed Hawkeye. This AI-powered SRE can quickly identify, diagnose, and resolve issues, freeing up human engineers to focus on more strategic work. In April, NeuBird raised a $22 million seed round from Mayfield. However, when Microsoft's venture fund M12 expressed interest in an investment, NeuBird couldn't resist. Since many of their customers run on the Azure cloud, this partnership has the potential to expand their market reach.Hawkeye works by using LLM reasoning to analyze logs from any system, including custom-built ones. As Rao pointed out, "LLMs have seen so many different application configuration scenarios that the fact that the LLM would run into an application log line message that it doesn't understand is minuscule." Hawkeye operates in a read-only mode, ensuring that it doesn't store any of the customers' proprietary data, which is crucial for banks and other organizations that handle personal information.
Hawkeye can continuously look for active alerts and alarms throughout the day. When it identifies an issue, it attempts to troubleshoot it. If it fails, it escalates the incident to a human engineer. The company has already attracted a diverse range of customers, from large car manufacturing companies and financial institutions to pharmaceuticals and even startups with as few as 30 employees and only one IT operations engineer who was struggling to keep up with incident tickets. While some are still in the pilot phase, many have moved to production mode in the last few months.
Competition in the Field
Although NeuBird has attracted significant investment and attention from venture capitalists, it's not the only startup working on AI-powered SRE tasks. Y Combinator has supported three such startups in 2024 alone (SRE.ai, Opslane, Parity), and other players like Cleric have also entered the market. Larger players like Moogsoft also offer automated incident response features.However, just like in sales automation and customer service automation, copilots or teammates, as Mayfield's managing partner describes NeuBird's offering, are making their way into many developer and DevOps functions. With the excitement from venture capitalists, NeuBird is definitely a startup to keep an eye on.