Machine Learning Engineer Technical Interview Questions & Answers (2026)

I'd follow a structured deployment pipeline: 1) Model packaging: serialize the model (ONNX for framework-agnostic portability, or native format for framework-specific serving). Package with dependencies in a Docker container, pinning exact library versions. Include a predict function that handles preprocessing (same feature transformations used in training). 2) Testing: unit tests for the predict function with known inputs/outputs. Integration tests against the feature pipeline. Shadow deployment: run the new model alongside the current one, compare predictions without serving the new model's results to users. 3) Serving: for real-time (REST API), I'd use FastAPI behind a load balancer with auto-scaling. For batch, scheduled Spark/Airflow jobs processing data in bulk. For edge cases with very low latency requirements, model compilation with TensorRT or ONNX Runtime. 4) Deployment strategy: canary deployment — route 5% of traffic to the new model, monitor prediction distribution and business metrics for 24 hours. If metrics are healthy, gradually increase to 100%. 5) Monitoring: track prediction latency, throughput, error rates, and crucially data drift (input feature distributions) and concept drift (prediction distribution shift). Alert when drift exceeds thresholds. 6) Rollback: one-click rollback to the previous model version. Model registry (MLflow) stores all versions with metadata for instant rollback.

A feature store needs two components: offline store for training and batch inference, online store for real-time serving. Offline store: I'd use a columnar format (Parquet on S3) organized by entity (user features, item features, interaction features). Feature computation runs as scheduled Spark/dbt jobs writing to the offline store. This is the source of truth for training datasets. Airflow orchestrates computation jobs. Online store: Redis or DynamoDB for low-latency lookups (<10ms). When features are computed offline, they're materialized to the online store. For real-time features (last 5 items viewed, current session duration), a streaming pipeline (Kafka → Flink) computes features in real-time and writes directly to the online store. Consistency: the critical challenge is training-serving skew. I ensure features are computed using the same code in both paths. Feature definitions are versioned and tested. Point-in-time correctness for training: when generating training data, features must reflect what was known at prediction time (no future data leakage). For 10M users at scale: Redis cluster with consistent hashing for the online store, partitioned by user_id. Read-heavy workload (100:1 read:write), so Redis handles it well. Feature computation is parallelized by user partition. I'd use Feast or a custom feature store built on these primitives.

There are two types of drift: Data drift: input feature distributions change (users' behavior patterns shift, new product categories appear). I detect this by comparing current input feature distributions against training data distributions using statistical tests (KS test for continuous features, chi-squared for categorical) or distance metrics (PSI — Population Stability Index). I'd monitor this daily and alert when PSI exceeds 0.2 for any feature. Concept drift: the relationship between features and the target changes (what makes users click evolves over time). This is harder to detect without ground truth labels. I monitor prediction distribution shifts and, when labels are available, track model performance metrics (AUC, precision, recall) on recent data vs the validation set. Response strategy: For gradual drift: scheduled model retraining (weekly or monthly) on a rolling window of recent data. The retraining pipeline runs automatically, evaluates the new model against a holdout set, and promotes it only if metrics improve. For sudden drift (after a product change, market event): triggered retraining with investigation of what changed. For feature drift: add drift-robust features, retrain with more recent data, or adjust feature engineering. I'd also maintain a dashboard showing model performance over time, feature importance stability, and prediction distribution — making it easy to spot degradation trends before they impact business metrics.

Single GPU training works for most models that fit in GPU memory (typically up to 12-80GB depending on the GPU). It's simpler, has no communication overhead, and is the right choice unless you hit memory or time constraints. Distributed training is needed when: 1) The model is too large for one GPU (large language models, large vision models), or 2) Training takes too long on one GPU (weeks instead of days). Data parallelism: same model replicated across N GPUs, each processing 1/N of the batch. Gradients are synchronized across GPUs after each step (all-reduce). Linear speedup in theory, ~80-90% efficiency in practice due to communication overhead. Use when: model fits on one GPU but you want faster training. PyTorch DistributedDataParallel is the standard implementation. Model parallelism: split the model across GPUs. Pipeline parallelism: different layers on different GPUs. Tensor parallelism: split individual layers across GPUs. Use when: model doesn't fit on one GPU (GPT-3 scale). More complex to implement and debug. Engineering challenges: communication overhead dominates at high GPU counts (gradient synchronization). Mixed precision training (FP16) reduces memory and speeds up computation. Gradient accumulation simulates larger batch sizes without more memory. I'd use DeepSpeed or FSDP (Fully Sharded Data Parallel) for practical distributed training, as they handle the complexity of sharding optimizer states and gradients automatically.

I'd design the A/B testing framework with these components: Traffic splitting: use a consistent hashing-based router that assigns users to variants deterministically (same user always sees the same variant for the test duration). The assignment is based on user_id + experiment_id hash, allowing multiple concurrent experiments without interference. I'd support percentage-based splits (90/10, 50/50) and segment-based targeting (test only on specific user cohorts). Metric framework: define primary metrics (the business metric we're optimizing — conversion, revenue, engagement), guardrail metrics (metrics that must not degrade — latency, error rate, user complaints), and secondary metrics (informational — for understanding the mechanism behind primary metric changes). Statistical methodology: I'd use sequential testing (not fixed-horizon) to allow continuous monitoring without inflating false positive rates. Power analysis before launch determines the required sample size for detecting the minimum detectable effect. For ML models specifically, I'd track both business metrics and model-specific metrics (prediction distribution, feature importance shifts) to understand why a model is performing differently. Engineering: experiment configuration in a central service (LaunchDarkly pattern) with feature flags. All metrics logged to an event pipeline and computed by a centralized experiment analysis service. Results dashboard with confidence intervals, p-values, and practical significance assessment. Safeguards: automatic shutdown if guardrail metrics degrade by more than a threshold.