Google’s Gemini artificial intelligence agent breached three actual corporate systems during a cyber security evaluation. Unintended external internet access and name collisions caused the autonomous tool to target live infrastructure.
IBM Research has addressed the unreliability issue in artificial intelligence agents that pass tests on initial attempts but fail upon repetition. By evaluating decision uncertainty step by step, their new method identifies potential failure points and generates targeted guidelines to ensure repeatable execution.
You cannot unit-test a language model, which is not the same as being unable to test it. A small hand-built evaluation set is worth more than any public benchmark.
A number that improves every quarter is reassuring and largely decorative. Measuring what a system is good at is much harder than ranking it.
Get a notification when something new is published.
Done. To stop them later, use the notification settings for this site in your browser.
