Research Scientist - Model Evaluation Job at Lumicity, San Francisco, CA

WktBcFFOWUZiU2FvRzZEeTJ5cFljVnplUEE9PQ==
  • Lumicity
  • San Francisco, CA

Job Description

AI Benchmarking & Evaluation Engineer

Join a team at the forefront of AI model evaluation, setting the standard for how large language models are tested and validated. In this role, you'll assess the latest AI models, design new benchmarks, and develop advanced evaluation methodologies. You'll work closely with engineers, AI researchers, and enterprise clients to ensure cutting-edge AI systems meet the highest standards. This role is a bridge between research and practical implementation and will suit someone who enjoys taking academic papers and creating working models.

Key Responsibilities:

  • Analyze and benchmark newly released AI models (DeepSeek, Gemini, etc.)
  • Develop and implement novel evaluation frameworks
  • Build datasets, manage labeling processes, and publish findings
  • Enhance automated evaluation techniques for AI-generated content
  • Collaborate with top AI labs and enterprise partners to refine best practices

Who You Are:

  • MSc or PhD from leading Computer Science or Machine Learning school
  • At least 3 years of experience in applied AI, with a focus on benchmarking or model evaluation
  • Strong background in designing evaluation methodologies
  • Passion for advancing AI assessment standards
  • Solid Python, PyTorch/TensorFlow and Django

Make a real impact in AI research and development—apply today!

Job Tags

Similar Jobs

Builders Warehouse

Class A Regional flatbed delivery drivers Job Job at Builders Warehouse

Class A Regional flatbed delivery drivers JobProfessional lumber and building material company looking for truck drivers to start immediately in our Aurora lumber yard. We are looking for reliable people with a Class A or B Commercial License to join our team delivering... 

Della Infotech, Inc.

ERM Analyst Job at Della Infotech, Inc.

 ...JOB SUMMARY ERM analyst provides analyses and recommendations to support risk owners and business groups. The successful candidate will collaborate with other ERM team members and business units on a variety of projects with a focus on risk identification, quantification... 

Shavano Family Practice

LIVE CHAT AGENT-WFH REMOTE Job at Shavano Family Practice

As a Live Chat Agent, you will be the first point of contact for customers seeking assistance via live chat. You will help customers with inquiries, resolve issues, and ensure they have a positive experience with our products or services. The ideal candidate is proactive... 

Halliburton

Service Supervisor I - Frac Acid Job at Halliburton

We are looking for the right people people who want to innovate, achieve, grow and lead. We attract and retain the best talent by investing in our employees and empowering them to develop themselves and their careers. Experience the challenges, rewards and opportunity...

Summit Medical Staffing Allied

Travel Nurse RN - Cardiac Cath Lab - $2,486 per week Job at Summit Medical Staffing Allied

 ...~ Shift: 8 hours, days ~ Employment Type: Travel HOW ABOUT THIS?177 beds Community Hospital Services include cardiology, neurology, pediatrics, ortho, otolaryngology, ophthalmology, psych, wound care, bariatric, sleep medicine, women's health, hospice, home health...