Researcher

Researcher

Researcher

Researcher

Researcher

In this work paper, we outline our vision for developing a tool ecosystem aimed at automatically testing AI-driven search engines in accordance with EU trustworthiness compliance requirements….Click on the PDF to continue reading.

In this session you will find a video of the first initial meeting of the full Trust4AI team.

GENIE facilitates the deployment and execution of Large Language Models (LLMs). This tool is specifically designed to integrate with MUSE, which generates test cases following a Metamorphic Testing approach, and GUARD-ME, which analyzes LLM responses to such test cases for the detection of possible biases.
To use GENIE follow the instructions in the github link below.

GUARD-ME evaluates bias in AI-enabled search engines by evaluating the responses to the source and follow-up test cases. It utilizes Large Language Models (LLMs) to detect any bias and ensure that these systems adhere to ethical standards. This tool is complementary to MUSE, which generates the test cases used, and GENIE, which facilitates communication with LLMs.
To use GUAR-ME follow the instructions in the github link below.

MUSE generates test inputs for testing the bias of AI-enabled Search Engines. It leverages the capabilities of Large Language Models (LLMs) to create a wide range of source and follow-up test cases. This tool complements GENIE, which manages communication with LLMs, and GUARD-ME, which checks for bias in responses from the systems under test.
To use MUSE follow the instructions in the github link below.

EVA takes LLM responses as inputs and classifies them as affirmative or similar. Its primary application is the assessment of large language model (LLM) responses during testing. EVA complements the POET component by evaluating the LLM responses obtained from the promps generated by POET.
To use EVA follow the instructions in the github link below.

POET is a template-based prompt generator for testing large language models (LLMs). It leverages customizable JSON templates to create diverse prompts, enabling comprehensive evaluation across various scenarios and inputs. This tool is complementary to EVA.
To use POET follow the instructions in the github link below.

In this post you will find examples of answers given by LLM GPT-3.5-Turbo to questions related to Safety. Click to see how the language models are still currently answering harmful questions about discrimination in the workplace and some minority groups.

In this post we show two examples biased by the Llama 3 language model. Click to see the full examples of machismo in everyday situations.