ML Evaluation Engineer

Dhaka (Hybrid) | Full -Time | 5+ years experience

Mission

Builds automated, sliced evaluation suites (by region, price tier, property type,

fairness proxies, outliers). Immediate impact: tells us exactly where the model

is weak drives accuracy gains and dramatically reduces risk on live→

valuation.

What You'll Own

● Evaluation Strategy & Standards

Define the end-to-end evaluation blueprint for our valuation models (and future CV

models): goals, metrics, stage-gates, statistical rigor, and reporting.

● Automated, Sliced Evaluation

Build production-grade evaluation pipelines with segment slices across region, price

tier, property type, fairness proxies, and outliers, plus time-based cohorts and data

freshness.

● Quality Gates for Release

Create pre-launch gating checks (offline + shadow/online) with clear go/no-go criteria,

confidence intervals, and power-aware significance testing.

● Calibration & Uncertainty

Implement calibration (reliability curves, ECE), prediction intervals (e.g.,

quantile/conformal), and coverage monitoring to reduce valuation risk.

● Error Analytics & Root Cause

Build error decomposition (MAE/RMSE/MAPE/WMAPE, residual analysis, tail risk),

hot-spot detection, and rapid drill-downs for "why" the model underperforms.

● Production Monitoring

Ship dashboards and alerts for data drift, target drift, segment regressions, and

fairness deltas; partner with MLOps to automate rollbacks/canaries.

● Model Cards & Exec Reporting

Publish consistent Evaluation Reports/Model Cards with headline metrics, slice

summaries, risks, and remediation plans that drive the roadmap.

Day-to-Day Responsibilities

● Stand up reproducible eval pipelines integrated with training and CI/CD (compare

champion vs. challenger automatically).

● Maintain curated test suites: time-split holdouts, stress/fuzz tests, synthetic edge cases,

and policy/guardrail tests.

● Partner with Data Engineering on data quality checks and with Backend/MLOps on

serving-side SLOs (latency, throughput, cost).

● Collaborate with Product & Head of ML to convert insights into prioritized fixes and

measurable wins.

● Prepare for computer vision: datasets, mAP/IoU metrics, class imbalance analyses,

and structured error taxonomies.

● Make yourself up-to-date with other competitors' prediction models.

Qualifications

Required

● 5+ years in ML Engineering / Data Science / MLE with a focus on evaluation,

measurement, or model quality.

● Strong stats toolkit: experiment design, hypothesis testing, power analysis,

confidence/credible intervals, and effect sizes.

● Hands-on with Python, Pandas/Numpy, Scikit-learn, SciPy/statsmodels; SQL for

slice definitions and cohorting.

● Experience building automated pipelines (e.g., Airflow/Prefect, dbt or equivalent), and

integrating with MLflow/W&B or similar.

● Deep familiarity with regression evaluation (MAE, RMSE, MAPE/WMAPE), calibration,

drift detection, and fairness slice reporting.

Nice to Have

● Conformal prediction / quantile regression; uncertainty estimation at scale.

● Tools like Evidently, Great Expectations, whylogs; BI/visualization (Dash/Plotly,

Streamlit, Looker/Superset).

● Exposure to CV metrics (mAP, IoU, PR curves) and dataset/label quality management.

● Domain experience in pricing/valuation or financial risk.

Salary & Benefits

• Competitive Salary: 200,000 – 250,000 BDT before taxes (Based on experience and expertise)
• Bonuses: 2 Eid Festival Bonuses per year
• Sports Allowance: Stay active and healthy with our support
• Free Food: Enjoy delicious complimentary breakfasts and lunches at the office
• Fun at Work: Access to indoor games and recreation
• Paid Leaves: Generous paid leave policy for work-life balance