Large Language Models (LLMs) like ChatGPT and Bard are all the hype right now, and rightfully. Recent advances have been astonishing, to say the least, and deserve to be celebrated. User numbers have sky-rocketed within days of the ChatGPT3 release in 2022, and social media has since been full of ‘prompt carousels’, advising how to make the most of this new technology, while of course AI ethics as a topic that has gained traction, especially since the recent letter signed by Elon Musk and others, calling for a ‘pause’ on generative AI developments, as well as the various reactions to said letter.
The big question for ‘learning’, however, is the following: “What real value do these models have for learning in and around the workplace?”.
And this question seems almost unjustified, after all we have been told that ChatGPT passes the Bar Exam, excels at the Biology Olympiad, and can even write and improve code, such as Python, while applications in Duolingo and the Khan Academy already see implementations for learning. So how could it not be all good news?
At the same time, we have also seen a huge recent trend around other AI applications and engines, particularly around truly adaptive programmes of learning, utilising the power of AI to personalise the learning journey, and offering a tailored delivery of content and questions, while adjusting in real-time to the learner’s current levels of knowledge, as well as each learner’s individual rate of progress.
So how do these compare? What are the strengths and weaknesses?
The issues with LLMs and chatbots
While LLMs excel as communication engines, and they very much do excel in this department, they are only as good as the knowledge engine that powers them. This has been covered previously, in How ChatGPT3 Impacts the Future of L&D in an AI World written together with Clark Quinn for the Learning Guild. This means that the better the system is trained on a topic, the better it will respond to prompts and questions around the topic, and with the internet as the training ground, we can see how these chatbots do very well on a huge range of general topics.
The first limitation, however, is that these systems are pure language engines. While they seem ‘logical’ at times, they have no understanding of the models or concepts they are discussing. A child that has overheard their parents discuss a certain subject time and time again will be able to quote sentences that imply ‘understanding’, when actually we know that the child didn’t ‘get’ what it was saying. In the same way, the chatbot system just knows how to talk about things. It doesn’t ‘know’ them.
Thus, these systems can talk about Pythagoras’ theorem, and seemingly explain the Maths, when actually the system is only copying how all the training it’s seen and been trained on ‘talks about’ Pythagoras. And this explains why recent advances, especially from ChatGPT3 to ChatGPT4, can herald fantastic progress on everything that is language based and doesn’t require logics, or building and working with models. Which is exactly what the Bar Exam is about, or the Biology Olympiad.
In the Physics, Maths or Chemistry Olympiad, where logics or models of the world are required, and especially new ones or ones on which the system has had very little training, the model underperforms, as can be seen from the data released by OpenAI with ChatGPT4. So until these models are also paired with a logics engine, results here will remain limited.
Furthermore, we identify four fundamental limitations for LLMs which negatively impact learning.
Number one, these models cannot utilise images for learning, such as highlighting a specific part of an electric circuit when explaining a circuit board. However, from the well-researched cognitive theory of multimedia learning, we know how important words and images are for learning.
Secondly, these models will change the text when they provide answers to questions. While these may be correct in most cases, it cannot be ruled out that responses are incorrect, which is a disaster for learning, especially in areas where risk comes into play, such as health & safety, cybersecurity, or compliance.
Thirdly, LLMs ‘hallucinate’, what’s also been coined the ‘bullshiting problem’, which is a reflection of a goal misalignment between prompt and what the user is aiming for. An LLMs goal is to produce text that follows the probability distribution of its training data that is most closely aligned with the prompt. The goal is not to produce the truth in response to the user’s query. And here is the issue. The system is like a friend that cannot ‘say nothing’. The system will answer the prompt or query with what is most likely going to be a good answer. It doesn’t know how good the answer is, or whether it is even true. We need to be aware that ‘great answers’ from chatbots are the outcome of a ‘coincidental’ alignment between the query and the training data.
And lastly, a key for effective and efficient learning is to uncover the ‘unknown unknowns’. If you didn’t ask, the system won’t tell you. Like that good friend who has been helping you to whom you say “but why did you never tell me that?”, and they say: “Well, you never asked.” A key skill for a good teacher, tutor or coach, is to uncover and guide the learner through the unknown unknowns.
To explain a certain topic area, or pretend to be a waiter in a Paris restaurant conversing with the learner in French on Duolingo, a chatbot is a great tool. But the aforementioned four fundamental limitations pose restrictions that we have to be aware of when it comes to the delivery of learning and training.
But how does this compare to adaptive learning engines?
The advantages of adaptive learning engines
Recent advances in AI technology have also given rise to truly adaptive learning experiences, where the AI takes the learner on a unique journey through a network of content, and guided by the instructional hierarchy – from easy to hard, and training the learner on the prerequisites before such knowledge is required for the more difficult topics.
Such adaptive experiences have found their way into a huge range of programmes of learning, since they can work with basically any format, from text via audio to video, delivering the content as it was designed, and ideally in line with the principles of multimedia learning.
The adaptation to each learner’s unique situation, prior and existing knowledge but also their unique pace of progress, allows for effective and efficient, asynchronous learning journeys. Continuous testing and practice, as well as spaced repetition, are all built-in – the good things that ‘make it stick’.
In addition, let’s compare this to the fundamental limits of LLMs and chatbots.
Firstly, images can be utilised, as designed by the Instructional Designer (ID). Thus, the signalling principle and the spatial and temporal contiguity principle, to name only two, can all be adhered to, in line with what we know about the theory of multimedia learning. Human learning is dual channel, and humans work best with sounds and images. Adaptive programmes of learning can deliver these.
Secondly, content is delivered not only the way in which it was designed, but also with the wording in place, as signed off by SMEs, and in line with regulations. IDs and learning leaders have, and retain, full control over what the learner sees.
Thirdly, since the content is delivered the way it was designed and signed off, there is no hallucination problem. The AI is just working with the content of a particular programme of learning, and not with any other inputs or training data. The AI is just measuring strengths, weaknesses and rate of progress of each learner, in a specific programme, and in regard to instructional hierarchy and learning objectives. No other training data is utilised, so again IDs and learning leaders retain full control over the content, and can ensure a bias-free and controlled learning environment.
And last but not least, adaptive learning will cover the content and the learning objectives, as per the scope of the programme, automatically. It’s job is to uncover the unknown unknowns for the learner, and guide them through the programme to certification. “But you didn’t ask!”, is not an issue in adaptive learning.
Professional certification bodies and training organisations are leading the way
As technology and innovation continues to progress and impress, Learning & Development (L&D) teams, especially in smaller organisations, are still busy getting their head around what all of this means, and how best the technology can be utilised.
However, we needn’t look further for an answer to the question of where things are heading than professional training organisations and bodies of certification, whose bread and butter is to train and certify professionals.
And here the trend is clear, given the aforementioned reasons. With effective and efficient training in mind, and the requirement to control the content and delivery, such organisations are implementing fully adaptive programmes of learning. After all, the competition is big in this space, and organisations delivering ineffective or old-fashioned one-size-fits-all training will be pushed out of the market swiftly by those adopting more innovative and impactful solutions.
OBRIZUM is the AI learning technology & data analytics company for Enterprise Businesses.
We leverage automation, adaptability and analytics to organisations to deliver adaptive learning experiences at scale.
Find out how OBRIZUM can help unlock the potential of your business by speaking to our team.
