Setting Up a Self-Hosted GitLab Runner on EC2 Linux

 GitLab provides its own runners to execute CI/CD jobs. However, if you want to run jobs on your own machine, GitLab allows you to set up a self-hosted runner. In this guide, we will go through the step-by-step process to set up a self-hosted GitLab runner and configure it to deploy a project.

Step 1: Configure a Self-Hosted Runner in GitLab

1. Go to the GitLab Homepage.
2. Navigate to your Group Settings and go to CI/CD settings.
3. Find the Runners section and click on Project Runner.
4. Click New Project Runner.
5. Assign a Tag (e.g., dev). This works like a label in Jenkins.
6. Click Create Runner. The runner is now set up in GitLab.

Step 2: Install the Runner on an EC2 Linux Instance

1. Click on “How to install” in the GitLab Runner setup page.
2. Copy the provided commands and execute them in your Linux terminal as a superuser:

• sudo su # Paste the GitLab-provided installation commands

3. Choose the appropriate architecture (e.g., amd64).

4. Run the installation commands.

5. Start the GitLab Runner service:

• gitlab-runner status # Check if the service is running

Step 3: Register the Runner with GitLab

1. Run the following command to register the runner:

• gitlab-runner register

2. Provide the required details:

• GitLab URL: Press Enter if GitLab is running on the default address.
• Runner Token: Copy and paste the token from the GitLab Runner settings.
• Executor: Choose shell.

3. Verify the runner:

• gitlab-runner verify

Step 4: Modify the Runner Configuration

1. Navigate to the GitLab Runner home directory:

• cd /home/gitlab-runner

2. View hidden files:

• ls -a

3. Open the .bash_logout file using vim:

• vim .bash_logout

4. Add comments to any necessary commands:

• # if # <command> # fi

5. Return to the main terminal:

• cd ~

6. Start the GitLab Runner:

• gitlab-runner run

Step 5: Create a GitLab CI/CD Pipeline

Write the pipeline in .gitlab-ci.yml format:

stages:
  - build
  - test
  - push
  - deploy

variables:
  DEPLOY_ENV: "production"
  GITLAB_USER_KEY: "this-is-my-secret-key"build_job:
  stage: build
  script:
    - echo "Building $CI_PROJECT_NAME the application"
  tags:
    - devtest_job:
  stage: test
  script:
    - echo "Testing in general build done by $CI_COMMIT_AUTHOR"
    - mkdir -p logs
    - echo "these are my test results" >> logs/app.log
  artifacts:
    paths:
      - logs
    expire_in: 1 week
  tags:
    - devdev_test_job:
  stage: test
  script:
    - echo "Testing in development using $GITLAB_USER_KEY"
  tags:
    - devprd_test_job:
  stage: test
  script:
    - echo "Testing in production using $GITLAB_USER_KEY"
  tags:
    - devpush_job:
  stage: push
  script:
    - echo "Pushing to Docker Hub for $DEPLOY_ENV"
  tags:
    - devdeploy_job:
  stage: deploy
  script:
    - echo "Deploying to EC2 for DOCKERHUB_USER"
    - echo "this log is from $CI_JOB_STAGE" >> logs/app.log
  artifacts:
    paths:
      - logs
    expire_in: 1 week
  tags:
    - dev

Step 6: Verify the Deployment

After running the pipeline, your project should be downloaded to your Linux instance. To check:

1. List directories:

• ls

2. Navigate to the GitLab Runner builds directory:

• cd /home/gitlab-runner/builds

3. Enter your project directory:

• cd 0/<project-name>

4. List the files to confirm successful deployment:

• ls

Conclusion

You have successfully set up a self-hosted GitLab Runner on an EC2 Linux instance, configured a CI/CD pipeline, and deployed a project. Now, your GitLab Runner will execute jobs directly on your server, providing more control over your deployments. 🎉