Baidu to Reimagine Search with ERNIE Bot; Tencent & ByteDance Join the ChatGPT Race; MOSS to be Open Sourced
Weekly China AI News from Feb 20 to Feb 26
Good morning, everyone! In this week’s newsletter, we’ll be discussing the latest announcements from Baidu on their upcoming conversational AI, ERNIE Bot. Additionally, ByteDance and Tencent are reportedly developing their ChatGPT-like technologies. Fudan University’s MOSS, which is similar to ChatGPT, experienced a surge of traffic causing it to be suspended since its debut. However, the good news is that MOSS will be open-sourced by the end of March.
Weekly News Roundup
Baidu Plans to Integrate ERNIE Bot Across All Operations, From Search to Autonomous Driving
What’s new: Baidu Co-founder and CEO Robin Li announced plans to embed its upcoming ERNIE Bot, a conversational AI bot built on Baidu’s large language models ERNIE, across all of the company’s operations, from search to autonomous driving, at the company’s latest earnings call.
We plan to embed ERNIE Bot into Baidu Search first. We believe this will reshape information generation and presentation, and create a new entry point for the next-generation internet.
In the future, we will also integrate ERNIE Bot into Xiaodu, which will allow Xiaodu to significantly upgrade all the smart devices and services.
We plan to make these technologies widely available to our customers, developers and ecosystem partners to help boost productivity across industries.
For intelligent driving, we plan to integrate ERNIE Bot into our auto solutions, further enhancing user experience.
Key takeaways:
- According to the earnings call, ERNIE Bot is set to be presented as a dialogue interface akin to ChatGPT in March. Baidu will integrate ERNIE Bot with its search and Baidu App, which is similar to the new Bing. However, it’s unclear whether ERNIE Bot and the new AI-powered Baidu search will meet the public together.
- Baidu executives believe ERNIE Bot will potentially add significant value to Baidu’s cloud computing as a “game changer”.
- ERNIE Bot will be better tailored to China’s market than ChatGPT as the underlying ERNIE models are localized AI foundation models.
Government support: Despite a recent report from Nikkei Asia claiming that Chinese tech companies will not be allowed to offer OpenAI’s ChatGPT services in the mainland, China’s Ministry of Science and Technology reiterated its support for AI and emphasized the huge potential of ChatGPT-like technologies in a news briefing last Friday. Read the Reuters story for more.
ByteDance Reportedly to Reveal Large Language Models in Mid-2023
What’s new: ByteDance, known for its popular social media platform TikTok, is expanding beyond being a social media company and entering the AI race by building large models that can process both language and images, according to Chinese media outlet 36Kr.
A ByteDance spokesperson stated that the relevant technology is in the early stages and not yet mature.
ByteDance has established teams dedicated to this task. The language model team is led by ByteDance’s search department, while the visual model team is led by the Intelligent Creation team under the Product R&D and Engineering Architecture department. However, both teams are still in the early stages of development and are not yet mature.
Mid-2023: According to a person familiar with the matter, the language model team was established this year with the primary goal of integrating the model with downstream businesses, such as search and advertising. The Search Business Department, AI Lab, and AML (Applied Machine Learning) team are all contributing to the language model team, which expects to launch its large language model in mid-2023.
Both the language and visual model teams report directly to TikTok’s technology lead, Wenjia Zhu.
Tencent’s Plans for ChatGPT Unveiled: Introducing HunyuanAide
What’s new: Tencent has reportedly formed a new project team, named “HunyuanAide”, to focus on the development of conversational AI products similar to ChatGPT. Tencent hasn’t confirmed the report yet.
Top AI talents: The team is led by Dr. Zhengyou Zhang, who is Tencent’s chief scientist and a mastermind in computer vision. Dr. Zhang will be assisted by Dr. Dong Yu, a distinguished scientist at Tencent who specializes in speech technologies and is an ACM/IEEE/ISCA Fellow.
Smart assistant: The HunyuanAide project team will work together with other internal teams at Tencent to develop a large language model using reinforcement learning algorithms for training. The aim is to improve Tencent’s smart assistant tools and create a smart assistant that can set the industry standard in China. Additionally, the team has been receiving sponsorships from Tencent’s business groups.
Fudan University to Open Source ChatGPT-Like Model In Late March
What’s new: Fudan University will open source its ChatGPT-like model, named MOSS, in late March, said its creator Dr. Xipeng Qiu, a Professor of Computer Science at Fudan University.
Who’s MOSS: MOSS, which made its public debut last Monday, is a conversational language model similar to ChatGPT, capable of performing various natural language tasks from question-answering to code generation. Its public debut last Monday garnered attention, with many calling it the first Chinese equivalent to OpenAI’s ChatGPT. However, it went down hours after its release due to a sudden surge of traffic and has since been suspended for an upgrade.
More about MOSS: Here’s what we know about MOSS based on Dr. Qiu’s recent speech at a Shanghai AI event and the public statement on MOSS’ GitHub.
- MOSS has around 20 billion parameters and was trained on publicly available Chinese and English data.
- MOSS is a home-grown model developed by Fudan University’s NLP lab, not built on open-source models like GPT-2/Bloom/OPT.
- The initial version of MOSS was called OpenChat in 2022, but the team quickly found out it was already in use.
- The MOSS team created both Chinese and English interfaces but found Chinese to be more challenging, with results not meeting their expectations. Therefore, they continued to collect Chinese data and train a Chinese model while mainly promoting internal testing in English.
- The MOSS team is confident that MOSS is closer to ChatGPT than other publicly available chatbots like Meta’s Blenderbot 3.
Trending Research
Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey
- Researchers from Pengcheng Laboratory, Anhui University, Peking University, and Sichuan University, gave a comprehensive survey of pre-trained large-scale models and hoped this paper could provide new insights and helps fresh researchers track the most cutting-edge works.
Zero-Shot Information Extraction via Chatting with ChatGPT
- Researchers from Beijing Jiaotong University and Alibaba’s DAMO Academy explored whether strong information extraction models can be constructed by directly prompting LLMs.
- Researchers from Peking University, Shenzhen Graduate School, Tencent, University of Macau, and Shenzhen Institute of Advanced Technology, proposed T2I-Adapter, a simple and small network that can provide extra guidance to pre-trained text-to-image models without affecting their original network topology or generation ability.
Noteworthy Stories
- The viral popularity of ChatGPT has stirred a frenzy within China where tech companies, battered by a two-year regulatory clampdown and the Covid-19 pandemic, have been seeking new sources of growth. — Wall Street Journal
- China Is Betting Big on Artificial Intelligence — Even as It Cracks Down on ChatGPT. — TIME
- According to a recent survey, 87 individuals have made significant contributions to the ChatGPT project, and 9 of these individuals are Chinese, including three who graduated from Tsinghua University. — 36kr
- Chinese views on ChatGPT and Baidu’s upcoming alternative. — Pekingnology
