QUALITY ANALYST PROFILE

AUDITING THE

INTELLIGENCE.

Contact Me

Handshake AI

AI Quality Analyst / LLM Evaluation Specialist

OCT 2024 — PRESENT Lead Quality Reviewer

CORE_SKILLS

NVIDIA OPENAI RLHF QA of QAs

English-Domain RLHF Auditing: Acted as the QA of QAs; auditing primary reviewer outputs and adjudicating disputed evaluations to ensure high-consistency for training data.

Autonomous UI Review: Performed QA on novel 3D modeling pipelines where AI operated professional software. Verified actions against high-fidelity recordings.

High-Stakes Logic Validation: Conduct second and third-pass reviews on LLM reasoning, validating logic and safety alignment.

Outlier AI

LLM & Multimodal QA Analyst

APR 2023 — PRESENT Evaluation Specialist

AREAS_OF_EXPERTISE

REASONING STEM LAW PSYCHOLOGY

Complex Reasoning Evaluation: Audited model outputs for RLHF and process-supervision projects in specialized domains.

Logical Coherence Auditing: Verified step-by-step reasoning in STEM, writing, and legal contexts.

Subjectivity Reduction Methodology

Observability Framework

General framework for turning abstract, subjective human data into high quality rubrics with concrete observable behaviors.

VIEW DOCUMENT

Interconnection of Research and Structured Analytic Techniques

Analytic Protocol

Decision making in high-risk environments using Heuer's CIA methodology. Teaching case study.

EXPLORE CASE

AI Red Teaming via fake RLHF Projects

Alignment Hacking

Conversation logs of LLM alignment hacking through prompt injection of a fake AI data training project.

ADVERSARIAL TRAINING LOGS

MASTER OF ARTS

Library & Information Science

University of South Florida | 2023

BACHELOR OF APPLIED SCIENCE

Organizational Administration

Pasco-Hernando State College | 2021

EMAIL dylanaoberry@gmail.com

LINKEDIN linkedin.com/in/dylan-o-berry

LOCATION Land O Lakes, FL, USA

AUDITING THE

INTELLIGENCE.

PROFESSIONAL EXPERIENCE