Mistral is a technology design and systems engineering company that offers complete solutions for product design and application deployment. The startup has now rolled out a new API for content moderation.
According to the AI company, safety is the top priority when using AI bots, and believes that system-level guardrails are critical to protecting downstream deployments. That’s why the company has released a new content Moderation API.
Moderation API uses the same API that powers the moderation service in Le Chat. It is trained to classify text in a range of languages, including English, French, German, and many more. And into one of nine categories: sexual, hate and discrimination, violence and threats, dangerous and criminal content, self-harm, health, financial, law, and personally identifiable information.
Aside from this, Mistral’s AI journey provides valuable insights into the intricate and dynamic nature of artificial intelligence. Along with this, it highlights the significance of good decision-making to drive the future of technology.
“Over the past few months, we’ve seen growing enthusiasm across the industry and research community for new AI-based moderation systems, which can help make moderation more scalable and robust across applications,” Mistral said in a blog post.
“Our content moderation classifier leverages the most relevant policy categories for effective guardrails and introduces a pragmatic approach to model safety by addressing model-generated harms such as unqualified advice and PII.”
Even though Mistral’s API is highly accurate, however, the company claims that it is still a work in progress. It is noteworthy that the company didn’t conduct a comparison of performance with popular moderation API, like Jigsaw’a Perspective API and OpenAI’s moderation API.
