Full job description Applied AI Scientist About KAPS Solutions: KAPS Solutions is a leading small business provider of business consulting services within the DoD and Intelligence Community. As a woman-owned small business, we promote a family-like atmosphere, access to executives, and team social engagements. We pride ourselves in teamwork and collaboration, work-life balance, and helping clients succeed at their missions. Join KAPS Solutions today and unlock your full potential! Location: mostly hybrid, Washington, DC Clearance: Public Trust Required Overall Assignment Description: The candidate will support the Advanced Research Projects Agency – Health (ARPA-H) by contributing expertise across the full lifecycle of AI system development: data aggregation and curation, model training and benchmarking, and deployment/validation in real-world clinical settings. The role requires the ability to critically assess technical approaches, advise program leadership, and ensure that funded performers and partners meet the highest standards of rigor, reproducibility, and real-world applicability. This is a Science and Engineering and Technical Advisor (SETA) role, focused on providing technical guidance and oversight. Responsibilities: ? Review and evaluate AI/ML technical proposals and deliverables from external teams. ? Provide guidance on model development, training methods, and validation strategies. ? Advise on integration of multimodal health data (EHR, imaging, genomic, and patient-reported). ? Assess alignment of model architectures and approaches with program goals and clinical use cases. ? Evaluate benchmarking results and provide feedback on methodological soundness. ? Advise on deployment considerations, including interpretability, reliability, and safety in real-world settings. ? Collaborate with clinical SMEs to ensure AI outputs align with clinician expectations and workflows. ? Identify risks, gaps, or weaknesses in technical approaches and recommend corrective actions. ? Support program leadership with technical assessments, reports, and recommendations. ? Produce high-quality written reports and presentations that synthesize complex technical findings for diverse audiences. Required skills and Demonstrated Experience: ? Advanced degree (Master’s or PhD) in Computer Science, Machine Learning, Data Science, or related field. ? 5+ years of experience in related field ? Experience applying advanced ML methods (LLMs, deep learning, etc) to complex health datasets. ? Familiarity with clinical and biomedical data types, including EHR, genomic, and imaging data. ? Demonstrated ability to review, critique, and guide AI/ML technical work at a high level. ? Strong understanding of evaluation methods for AI/ML, including robustness, fairness, interpretability, and reproducibility. ? Strong written and visual communication skills, with ability to produce high-quality reports and presentations. ? Experience in healthcare AI or biomedical applications. ? Familiarity with the rare disease field. ? Prior involvement in technical advisory, evaluation, or SETA-style roles. ? Experience working with interdisciplinary teams including clinicians and patient stakeholders. ? Experience working in an early-stage, fast-paced, startup environment. Salary: $170k-$190k Job Type: Full-time Pay: $170,000.00 - $190,000.00 per year Benefits: 401(k) 401(k) matching Dental insurance Employee assistance program Employee discount Health insurance Health savings account Life insurance Paid time off Parental leave Professional development assistance Referral program Retirement plan Tuition reimbursement Vision insurance Education: Master's (Required) Experience: AI: 5 years (Required) Work Location: Remote