This Week in Edtech 3/24/23
The Open Beta Era, Upcoming EdTech Insiders Events, Khanmigo, ChatGPT 4, and more!
STORY: EDTECH IN THE AGE OF THE OPEN BETA
This week we have an article written by Edtech Insiders co-founder Ben Kornell on AI, the current model of open beta releases, and what this means for EdTech as AI continues to permeate and become more relevant in the industry.
With Google’s “experimental” release of Bard last week, many are gleefully declaring that “the AI Wars have officially begun”. Whatever the tagline, AI is clearly powering a Cambrian explosion of new tools and capabilities in 2023, and companies are hell-bent on bringing generative AI to the masses.
This moment is profound— not only in WHAT companies are launching but also HOW.
The AI wars have created a “release or die” mentality across tech, where highly produced product launches (think Steve Jobs on a stage) – have been replaced by the “open beta.”
The playbook works like this: companies get a workable prototype, announce a cascade of caveats, slap on a “for research only” warning, and then release their product to the masses. As a user, it can feel incredibly thrilling to see open betas hit day after day, and the shift towards this release strategy has certainly supercharged social media-fueled buzz: Who is releasing what? What are the newest tools and hacks? What did that AI say?!?!
In fact, open beta testing is an age-old product strategy that is finally getting its go-to-market moment. The concept actually originated at IBM in the 1950s as industry partners would do field-testing before full global release. The popularity of closed and open betas grew in the early 2000s in gaming, where a dedicated group of gamers could get early access to a game in exchange for finding all of the bugs and errors in it. While this helped build a cult following for a product, it was also an essential tool for software developers to improve the product.
As the agile software development philosophy has gained traction, the open beta concept has accelerated to the point that today, the concept of a traditional “product launch” could be considered more of a marketing construct. Products now go through alpha, beta, and global releases in real time, with product teams shipping new and updated features on monthly, if not weekly, timelines.
Google Bard and the Mainstreaming of Open Betas
That said, last week marked a true turning point in the mainstreaming of open betas.
Over the last decade, Google has become the archetypal Big Tech company whose every lumbering move is dissected by pundits, users, and (especially) lawyers. Gone is the 20% “free-time” ethos that defined the company’s innovation engine in the early 2000s. Google used to be the ones doing open betas; now that they are encumbered by investigations and lawsuits, they are the incumbent juggernaut.
Which is why last week’s Google Bard release was both so clearly painful and also powerful.
With little fanfare, the Bard release popped up in my feed on Tuesday morning and as I clicked to the Bard home page, the first use case was laugh-out-loud funny: “Bard can explain why large language models might make mistakes.” Of course, all users will want to ask that first! Ha!
As I completed my journey from waitlist to user, the disclaimers continued like a drumbeat, always with the theme “Bard is an experiment” and “don’t rely on Bard’s responses.”
While I’ll leave it to others to psychoanalyze the corporate messaging revealed in this Google release, the key takeaways are fully aligned with the current open beta ideology:
It may not work
It may be dangerous
It’s not our fault
With Google’s release, the abdication of all responsibility for product dysfunction and potential harm has now become universally accepted as the price of admission for any AI tool… and that is a legitimate cause for concern, especially in Edtech.
Risks of Open Beta
Recent articles have highlighted just a few of the dangers of the open beta approach. Kevin Roose chronicled the emergence of “Sydney,” a home-wrecking ChatGPT persona hiding inside of Microsoft Bing’s new AI-powered search engine. For social media giant Snap, the rushed release of its AI bot quickly led to advice about how teens can avoid getting caught for drugs or booze. In just a few months, we already have a catalog of examples where AI bots have amplified misinformation and bias. With enough tuning and training, could the hallucinatory or bias risks have been mitigated?
Another, less appreciated risk for these open betas is the fact that developers are often using other, sometimes non-reliable, software to shortcut the product launch. Many of the tools that help accelerate an open beta aren’t ready for millions upon millions of users, and the underlying backend architecture is also under constant stress to scale with user demand. Just last week ChatGPT was out for several hours as a security breach caused by open-source software resulted in chat histories showing up in other people’s feeds.
What does this mean for those of us in tech- and specifically in Edtech?
Which brings us to Edtech, where we have a similar AI race underway.
Startups seem to pop up on a daily basis touting reading, writing, math, and lesson-planning AI. Quizlet, Duolingo, and Khan Academy are just a few of the big players diving headfirst into the AI revolution. So far, these organizations have demonstrated some real caution around the “open beta” approach- which makes sense, because use cases for kids and managing the data kids are creating must be treated like a whole different ball game.
First, there’s COPPA, which requires parental consent, notice of data collection techniques, parent access and control, confidentiality and security, and data retention policies that protect kids’ anonymity. These guidelines have always been a unique and important part of the Edtech pact with families and educators, but with ChatGPT, Bard, and other consumer products entering the edtech stack as “research experiments,” it raises real and deep questions about COPPA in an AI open beta world.
Second, beyond regulations, there is the whole issue of parental and educator trust. An open beta that gives a crazy answer to an adult can come across as funny or at least simply off-base. When that same AI bot provides a crazy response to a child, the impact can be quite concerning. Products in Edtech live or die based on two key factors: Is the product safe and does the product reliably work? That is a stark contrast to the expectations and realities of an open beta AI movement.
What do we do about this?
Ultimately, this story is writing itself in real time, much like… a conversation in the ChatGPT box. For Edtech entrepreneurs, investors, and advocates, some initial advice would be:
Take a beat: Realistically how important is it to be first-to-market? If the past is any guide, Edtech is a marathon, not a sprint, and the best quality products win over time. Commercial products are also evolving so quickly that your first-to-market capability may be quickly subsumed anyway. Play the long game and know your users!
Consider closed betas: When everyone goes right, go left! Closed betas can actually build buzz too, and they accomplish many of the same product objectives. Closed betas allow you to onboard users and leverage human communication to ensure guardrails.
Tune tune tune: Work with commercial LLMs to better tune generative AI for kids’ use cases. This can include supervised learning on top of the LLM, adding fairness constraints, adversarial training, and using re-sampling methods. Your defensibility is ultimately in your tuning and your user experience.
Keep humans in the loop: Products that work WITH parents/educators rather than around them can establish trust and long-term success in Edtech. Along with this, shifting from ‘answer’ to ‘suggestion’ can be a key product move. AI recommendation-engines can be just as helpful, if not more, because they put some of the cognitive lift on the human.
Identify appropriate age use: If your AI bot is only safe for kids over a certain age, then say so clearly from the jump. We don’t show rated R movies to 8 year olds for a reason.
Ultimately, this AI open beta moment regularly vacillates between thrilling and terrifying, but the underlying potential to transform learning access and outcomes is huge. That said, if we’re not careful, the AI moment in Edtech could be undermined by the open beta moment in tech.
UPCOMING EDTECH INSIDERS LIVE EVENTS
We have two upcoming in person Edtech Insiders events, one in The Bay Area and one in San Diego! We would love to see you there, please RSVP on Eventbrite if you plan to attend!
PODCAST DEEP DIVE: KHANMIGO WITH KRISTEN DICERBO
Last week we had an amazing conversation with Kristen DiCerbo, the Chief Learning Officer at Khan Academy! Together we discussed Khanmigo, Khan Academy’s new AI based offering, AI in education, and how this shift is changing the industry.
This podcast episode was so rich we wanted to share out some key quotes, and encourage you to give it a listen if you haven’t yet. Kristen is an expert in the online learning industry, and getting a behind the scenes look at Khanmigo, what it is now, and how it came to be was a simply illuminating conversation.
Enjoy some of these selected quotes to give you a taste of what we talk about together, and listen to the podcast episode for more!
“If you ask students what they want, there's a lot of places where they can go to get the answer to homework that they want to get done. And so we have a little more space to say, that's not who we're going to be, we don't have to be exactly meet what this market, the student market, wants, because we've got revenue that isn't tied to the market, but more tied to what philanthropists and other folks want.”
- Kristen DiCerbo, Chief Learning Officer at Khan Academy
“All of the chats and the chat history are recorded and are available to the students. So the student if they said, “Hey, I had a good chat about this kind of problem two days ago,” can go back and see that. But it's also available, if they have a parent or a teacher linked to their account, it's available to them. So they can see what the history is. The next step, as you can imagine, is using the AI to summarize those chats for the teacher or parent… You take the chat histories, and you say, hey, teacher, hey, parent, here's what so and so was working on.”
- Kristen DiCerbo, Chief Learning Officer at Khan Academy
TOP EDTECH HEADLINES
ChatGPT4
The release of ChatGPT 4 has led to the creation of a series of new EdTech tools. Explore new tools that have recently been announced below, and listen to our recent podcast episode to hear Kristen DiCerbo, Chief Learning Officer at Khan Academy, chat about Khanmigo.
Other Edtech using ChatGPT:
We also saw a flurry of non-education AI-powered products and co-pilots, such as Google Bard, Adobe Firefly, and Microsoft Loop.
Pearson Sells OPM Arm to Private Equity Firm
Pearson, once the leading player in the online program management industry has sold its OPM arm. This is one indicator about what’s going on in this changing industry, and a symptom of further regulations making it difficult for edtech companies to provide services to colleges and universities.
Post-COVID Edtech
Public schools struggled to keep up during the pandemic, and edtech was one tool used to address this issue. What pandemic era edtech solutions are here to stay? What is already gone? What is still needed to come back in the aftermath of covid learning losses and setbacks? Explore the following articles from this week on how this problem is currently being addressed:
Public School Enrollment Dropped by 1.2M During the Pandemic
Federal Funding Won’t Be Enough to Help Students Catch Up Post Pandemic
Skills Development in the Spotlight
The focus on proving career skills, closing the skills gap, and gaining practical knowledge from higher education has been growing in recent years, and appears to be reaching a peak as employers, students, and education institutions are pivoting to meet this need.
OPEN EDTECH ROLES
Edtech mentoring company Curious Cardinals is hiring for several roles, check them out at their dedicated career board! We are interviewing Curious Cardinals founder Audrey Wisch in an upcoming episode of the podcast, and we love the work they are doing!
FUNDING
ACQUISITIONS
PARTNERSHIPS
LISTEN
Catch up on recent episodes of the Edtech Insiders podcast!
Thanks for reading Edtech Insiders! Subscribe for free to receive weekly updates and support the podcast.