Continuous Integration (CI)

Idea of this post was appeared when i needed to implement an automated process of deploy. 99% of guides is about basic implementation and general tips.

In this section, we discuss how to optimize your CI pipeline for better performance and reliability.

1. Linter

Your project must adhere to the style guide. Speed up the process by linting only the files that have changed.

# Find any changed ".rb" files compared with $TARGET_BRANCH variable.

CHANGED_FILES=$(git diff --name-only origin/$TARGET_BRANCH...HEAD -- '*.rb')

if [ -n "$CHANGED_FILES" ]; then

echo "$CHANGED_FILES" | xargs bundle exec rubocop --force-exclusion --fail-level A || true;

else

echo "No changed files.";

fi

2. Tests

Ensure your tests are reliable and consider parallelizing your tests for faster execution.

3. Vulnerability Audit

Check for project vulnerabilities regularly but adjust the audit process to avoid interfering with development based on the project's risk level.

4. Build Check

Ensure your project can be built in production mode, although most errors will likely be caught by the linter and tests.

CI Best Practices

  • Parallelize your steps for faster CI processes.
  • Cache dependencies to avoid repeated downloads.
  • Optimize dependency installation (e.g., using `npm ci` for javascript projects).
  • Break large CI processes into smaller stages.

Secrets Management

Use secrets management to store sensitive information like API keys and passwords securely and confidential.

Apply the DRY (Don’t Repeat Yourself) principle by reusing secret variables across your pipeline without exposing them in logs.

Continuous Deployment (CD)

CD allows you to deploy your applications with a single button click. It should be safe, stable, and simple to use.

1. Custom User and Group for Deployment

Create a custom user and group on your server for deployment. This user should not have root access and should only be able to restart the project and check its status.

2. Git-based Deployment

Ensure that your project is updated using git to avoid stale files from old branches.

git reset --hard HEAD

git fetch origin "$TARGET_BRANCH"

git checkout -B "$TARGET_BRANCH" "origin/$TARGET_BRANCH"

git reset --hard "origin/$TARGET_BRANCH"

git clean -fd

3. Local Testing of Deployment Steps

Making sure that your deployment steps are correctly could be possible using bash scripts before running them in the pipeline. This can save you time by catching errors early.

4. Server Connectivity Check

Check connectivity to the server to avoid issues with SSH or server access during deployment.

ping -c 3 "$SERVER_IP" || (echo "Ping failed" && exit 1)

5. Fail Pipeline on Errors

Use `set -e` and `set -o pipefail` to stop the pipeline immediately if any step fails.

set -e

set -o pipefail

CD Best Practices

  • Ensure deployment is safe, stable, and simple.
  • Use a custom deploy user with limited access to reduce risk.
  • Test each deployment step locally to prevent mistakes in the pipeline.
  • Ensure the server is accessible before deployment starts.