Researchers at OpenAI published a paper last weekend hypothesising that hallucinations occur because of the pre-training process of AI models. So, what should you know about it?
Why it happened
For your information, this paper focuses on getting the model to correctly predict the next word, regardless of whether it is right or wrong. The model is trained to provide answers of any kind, with no incentive to respond with "I don't know."
In the model evaluation, the AI is given high marks for correct answers. In situations where it has to answer a question, the AI will choose to guess an unknown answer with a probability of being correct rather than choosing not to answer and getting a score of zero.
What’s next?
Moreover, researchers are now proposing that in model evaluation, incorrect answers will receive negative marks, while incorrect answers given with confidence due to hallucinations will be given a higher mark deduction.
Besides that, half a mark will be given if it admits to not knowing. This way, the AI model being trained has an incentive to give the right answer all the time. In the future, the problem of hallucinations could be reduced with better AI model training systems.
Did this news catch your attention? Stay tuned for more news like this at TechNave!
COMMENTS