LLM Prompt eval
LLM Prompt Eval Tool is an app that enables developers at Pythagora to evaluate prompts on different LLMs multiple times in parallel. You enter a prompt, select the LLMs you want to test, and specify how many times you want to run it. The tool executes the prompt on the chosen LLMs the desired number of times and displays the results. You can also create a review prompt that evaluates the responses and shows the success rate for each selected LLM model. This is an invaluable tool that all developers at Pythagora use daily.