Snap Judge

Probabilities, not prose.

Write questions in TypeSafe Jev's three types - yes/no, pick one, pick a level - and run them on Jev or on any LLM. Jev returns a distribution in one call. An LLM returns one answer, so Snap Judge asks it several times and counts the votes, showing how sure each model really is and how often it answers outside the options you gave.

Independent project, not affiliated with TypeSafe AI. Your keys stay in this browser and go only to the provider you pick.

Ask

Add
Code for the real TypeSafe API 

      

    


    

      

        
Run on

        
      


      


      

        Sampling, thresholds and the local proxy
        

          
          
          
          
          
          
          
        

        
Thresholds follow TypeSafe's confidence-routing pattern: below the floor a person decides, above "act" the code acts, between them it asks to confirm.

        

          
          
        

        
TypeSafe, NVIDIA and Cerebras block browser requests. Run node proxy.mjs from the repo, then open 127.0.0.1:8787 - every provider works there.

        
        
Off: keys last until you close this tab. On: they are kept in this browser's storage for this site.

      


      

        
        
      


      

    

  


  

    
How it works

    

      

        
Jev

        
One request, every question answered in parallel, each answer constrained to your options with a probability for every one. The confidence shown for Jev is TypeSafe's own number.

      

      

        
An LLM

        
Each sample asks all the questions at once and must reply with JSON. Snap Judge checks every answer against its options; anything else is a type violation and is left out of the counts. The share of valid votes per option becomes the distribution. With five samples every probability is a multiple of 0.2.

      

      

        
Agreement

        
For LLMs, "agreement" is our measure: 1 minus the normalised entropy of the votes. TypeSafe does not publish how its confidence is computed, so the two are not the same number. Compare the shapes of the distributions, not the digits.

      

      

        
Your keys

        
Keys go straight from this page to the provider. The only exception is the three providers behind the local proxy, which runs on your machine, listens only on 127.0.0.1, forwards only to those three hosts, and never logs a key.