Note: The job is a remote job and is open to candidates in USA. Skill is a tech real estate company seeking licensed real estate agents located in Seattle, WA to work part-time as Agentic AI Judges. The role involves reviewing AI/ML processes related to verbal interactions between home buyers and agents, ensuring quality standards, and collaborating with engineers and data scientists to optimize model performance.

Responsibilities

• Review and/or evaluate annotated data to ensure it meets quality standards and project requirements
• Provide feedback and guidance to engineers to improve accuracy
• Collaborate with engineers and data scientists to optimize model performance based on annotated data
• Develop and maintain annotation guidelines and best practices
• Provide training data to engineers to establish Ground Truth data
• Provide input/examples on Training content
• Identify gaps in training content and suggest improvements

Skills

• Must have an active Real Estate License, or previously been licensed with experience selling real estate in Seattle
• Minimum of 3+ years of active experience as a licensed real estate agent or broker, demonstrating a deep understanding of the real estate transaction process
• Excellent attention to detail and analytical skills
• Ability to work effectively in a collaborative team environment
• Demonstrated deep interest and curiosity in Artificial Intelligence, including experience reviewing or evaluating AI-generated content
• Experience with copywriting, User Experience authoring, and content development around voice and brand guidelines
• Experience with Machine-generated Speech evaluation, Search Results Evaluation, Search Query Categorization
• Experience following or managing complex processes, and working both independently and with a team to achieve a goal
• Experience with tools and software used for Natural Language Processing annotation is a plus
• Experience working on Natural Language Processing projects with a focus on transcriptions, localization, and translation
• Bachelor's degree in English, Linguistics or a related field

Company Overview