Quality Gates

This guide explains how to keep quality gates passing and documents common fixes for build and test issues.

Overview

Quality gates ensure code quality before merging. Our pipeline checks:

Running Quality Checks Locally

Always run quality checks before pushing:

# Full quality check
make all

# Or step by step:
./build.sh    # Build all packages
./test.sh     # Run all tests

Common Build Issues and Fixes

1. Incorrect Import Paths

Problem: Import errors like ModuleNotFoundError: No module named 'mlflow_sklearn.doe_library'

Root Cause: Shared libraries (doe_library, io_library, etc.) are standalone packages, not submodules of other packages.

Fix: Use direct imports from the shared library:

# ❌ Wrong - treats doe_library as a submodule
from mlflow_sklearn.doe_library.clip_float import clip_float
from mlflow_sklearn.io_library.read_from_s3 import read_parquet_from_s3

# ✅ Correct - imports from standalone library
from doe_library.clip_float import clip_float
from io_library.read_from_s3 import read_parquet_from_s3

Files commonly affected:

• mlflow_tf/utils/doe/*.py
• mlflow_spark/doe_library/*.py
• sagemaker_sklearn/*.py

2. Missing Dependencies

Problem: Build fails with ModuleNotFoundError for third-party packages.

Fix: Ensure dependencies are listed in the appropriate file:

# For packages with setup.py/pyproject.toml
# Add to install_requires in setup.py or dependencies in pyproject.toml

# For scripts without packaging
# Add to requirements.txt in the package directory

3. Build Order Dependencies

Problem: Application packages fail because they depend on library packages that haven’t been built yet.

Solution: The build script (build.sh) handles this automatically by building in dependency order:

1. Library packages first: doe-library, io-library, metrics-library, feature-library, plot-library
2. Application packages second: mlflow-sklearn, mlflow-tf, mlflow-spark, etc.

If you add a new package, add it to the correct section in build.sh.

4. Scala Build (Optional)

Problem: sbt: command not found

Impact: The mlflow-scala package is skipped (warning, not failure).

Fix (if you need Scala):

# Install sbt
sudo apt install sbt

# Rebuild
./build.sh

Common Test Issues and Fixes

1. Missing Test Fixtures

Problem: fixture 'xxx' not found

Fix: Ensure fixtures are defined in conftest.py:

# tests/conftest.py
import pytest

@pytest.fixture
def sample_data():
    return {"feature1": [1, 2, 3], "target": [0, 1, 0]}

2. Import Errors in Tests

Problem: Tests can’t import the package being tested.

Fix: Ensure the package is installed in development mode:

cd src/your-package
pip install -e .

3. Test Discovery Issues

Problem: pytest doesn’t find your tests.

Fix: Follow naming conventions:

• Test files: test_*.py or *_test.py
• Test functions: test_*
• Test classes: Test*
• Test directory: tests/ or test/

Static Analysis (SonarQube)

Configuration

The project uses SonarQube for static analysis. Configuration is in sonar-project.properties:

sonar.projectKey=mlops-with-mlflow
sonar.sources=src,docs
sonar.cpd.exclusions=**/tests/**,**/test_*.py
sonar.python.coverage.reportPaths=coverage.xml

Common SonarQube Issues

Generating Coverage Reports

# Generate coverage.xml for SonarQube
pytest --cov=src --cov-report=xml

# View HTML report locally
pytest --cov=src --cov-report=html
open htmlcov/index.html

IDE Configuration

After building, your IDE might still show import errors. This is a cache issue, not a real problem.

PyCharm / IntelliJ

1. Mark source roots: Right-click src/ → “Mark Directory as” → “Sources Root”
2. Invalidate caches: File → Invalidate Caches → Invalidate and Restart
3. Check interpreter: Ensure using the correct virtual environment

VS Code

1. Select Python interpreter: Ctrl+Shift+P → “Python: Select Interpreter”
2. Reload window: Ctrl+Shift+P → “Developer: Reload Window”
3. Check Pylance settings: Ensure python.analysis.extraPaths includes src/

Troubleshooting Checklist

When builds or tests fail, check these in order:

□ Are all library packages built first? (run ./build.sh)
□ Are imports using correct paths? (not submodule paths)
□ Is the virtual environment activated?
□ Are all dependencies installed?
□ Did you add new files to __init__.py exports?
□ Are test fixtures defined in conftest.py?
□ Do test files follow naming conventions?

Adding New Packages

When adding a new package:

1. Create the package structure:

   src/my-package/
   ├── my_package/
   │   ├── __init__.py
   │   └── main.py
   ├── tests/
   │   ├── __init__.py
   │   └── test_main.py
   ├── setup.py (or pyproject.toml)
   └── requirements.txt
   

2. Add to build.sh: Edit the LIBRARY_PACKAGES or APP_PACKAGES array.

3. Verify locally:

   ./build.sh && ./test.sh
   

4. Update documentation if needed.

Summary of Past Fixes

See BUILD_FIX_SUMMARY.md in the repository root for detailed fix history.