Sunday, 31 March 2024

Docker Cloud Agent in Jenkins

1. What to use Docker Cloud Agent for?
Docker Cloud Agent is a agent of Jenkins to clone source, build, test for CI.

2. How to use Docker Cloud Agent?
- Setting up Docker cloud Agents

3. What does Docker Cloud Agent include? 

--------------------------
Detailed instructions

* Setting up Docker Cloud Agents (1)
- Prerequisite: Docker installed
- Prepare to connect docker in host
+ Install Plugin cloud provider/ select plugin: Docker/ Download now and restart after install.
/ There are many plugins for Cloud providers: Docker, Kubernetes, Amazon EC2, Azure VM Agents,...
+ Build and run socat container (2) to connect Jenkins with Docker (running jenkins as container, the container can't reach docker host port).

docker run -d --restart=always -p 127.0.0.1:2376:2375 --network jenkins -v /var/run/docker.sock:/var/run/docker.sock alpine/socat tcp-listen:2375,fork,reuseaddr unix-connect:/var/run/docker.sock
docker inspect <container_id> | grep IPAddress

- Connect docker in host: Dashboard/ Manage Jenkins/ Manage nodes and clouds/ Configure Clouds/ (Go to plugin manager to install Plugin cloud provider)/Add a new cloud/ Select Docker/ Docker cloud detail/ Docker Host URI: tcp://<IPAddress_Socat>:2375/Check Enable/ Test Connection.
- Create Docker Agent templates (can create multiples): Add Docker Template/ Labels: Docker-agent-alpine/ checked Enabled/ Name: Docker-agent-alpine/ Docker Image: jenkins/agent alpine-jdk11/ Instance Capacity: 2/ Remote File System Root: /home/jenkins/ Save
+ Labels: for jobs select which agent should run the build.
+ Instance Capacity: is how many actually spawn.
+ Remote File System Root: workspace of Jenkins is going to get created.
- Using Agents in job: Select job/ Configure/ checked Restrict where this project can be run?/ Label Expression: Docker-agent-alpine/ Save/ Build job again/

Reference
- (1) (Setting up Docker Cloud Agents 00:33:40 Setting up Docker Cloud Agents)
- (2) Build and run socat container

Implementing CI/CD for a Laravel App with Jenkins Using Docker Cloud Agents

1. Architecture
- Source Repository for store source code (Github)
- Jenkins server for master server handle Pipeline CICD (using local)
- Docker host to clone source, build, test for CI (using local)

- Server to deploy Laravel app for CD (VPS || Cloud)

2. Install tools & prepare source code
- Prepare source code Laravel
- Install Docker in local
- Install Jenkins using Docker in local: Using Docker plugin | Docker pipeline plugin
- Setup Docker cloud agent for CI (to clone source, build, test)
- Setup Permanent Agent server (using VPS) for CD (to deploy Laravel app)
||
Cloud Agent server (using AWS||GCP||Azure) for CD (to deploy Laravel app)

3. Setup Pipeline CICD
- Create job DeployLaraveApp
+ Type: pipeline
+  Trigger: schedule to check source change || when source code change (case Jenkin server not using in local by using webhook in github)
+ Using jenkinFile
- Run build when trigger happen.

-------------------------
Detailed instructions

* Prerequisite
- Install Jenkins, Docker

* Source Repository (Github)
- Source Laravel
- dockerfile/ docker_compose
file
- Jenkinfile

* Setup Docker cloud agent for CI (to clone source, build, test)
+ Using: Docker Pipeline Plugin in Jenkins
Or using: Docker Cloud Agent in Jenkins

* Setup Permanent Agent server (using VPS) for CD (to deploy Laravel app)

* Create job DeployLaraveApp

* Using Ngrok for Jenkins in localhost: https://ngrok.com
- run: ngrok http http://localhost:8090
- endpoint page: https://dashboard.ngrok.com/cloud-edge/endpoints

* Setting webhook github for Jenkins in localhost

- When create Pipeline:.../ Build Triggers: select Github hook trigger for GITScm polling.

- Add webhook in Github: Select Repository/ Setting/ Webhooks/ Add webhook/ Payload Url: enter url get from Ngrok (for jenkins localhost)/: {JenkinServer}/github-webhook/ / Content type: application/json / Which event would you like to trigger this webhook: Let me select individual events: Enable Pull request, Pushes.. / Add webhook.

* Set permission for jenkin handle with /var/www

- Copy source code from workspace in jenkins to VPS server

cp -r * /var/www/html

- When install nginx/ apache2, then auto create group www-data
- or using this: groupadd www-data
- Add user jenkin to this group: 

sudo usermod -aG www-data jenkins
sudo newgrp www-data // to switch to www-data group for current user (because one user can has many groups)

- Change owner directory, change permission

sudo chown jenkins:www-data -R html
sudo chmod -R 2771 html 
// 2: is the set-group-id: ensure that the created file in html directory inherit the group ownership of html directory

Thank you.

Tuesday, 26 March 2024

Pipelines in Jenkins

1. What to use Pipelines for?
- Pipeline can be considered a type of job
- Pipeline is more advanced and flexible way to define the CI/CD process compared to traditional job types.

2. How to use Pipelines?
- Create a Pipeline: Setting up Declarative pipelines using Groovy
+ New Item/ Name: first_build_pipeline/ select Pipeline
+ Pipeline Script: can get Jenkins file from Git | directly enter script
>> get template pipeline here

3. What does Pipelines include?

Reference

Thank you




Jobs, Builds & Triggers in Jenkins

1. What to use Jobs, Builds & Triggers for? 

- Jobs is individual tasks or workflows that need to be executed. These tasks could include building, testing, deploying, or any other automation task.

- Builds is the execution of a job. When a job is triggered Jenkins initiates a build process to perform the defined tasks within that job.

- Triggers in Jenkins determine how and when a build should start:

+ Build after other projects are built: This trigger allows a job to be executed after one or more specified projects have completed their builds. It's useful for setting up a sequence of jobs where the output of one job is required before starting another. For example, you might have a deployment job that should only run after a successful build job

+ Build periodically:With this trigger, you can schedule jobs to run at regular intervals, using a cron-like syntax to specify the schedule. It's ideal for workflows that need to occur regularly regardless of code changes, such as nightly builds or weekly integration tests.
GitHub hook trigger for GITScm polling:

+ This trigger is designed for projects hosted on GitHub. It utilizes webhooks to start builds automatically when a change is pushed to the repository. Instead of polling GitHub for changes, which can consume resources and delay builds, the webhook immediately notifies Jenkins of any SCM changes, triggering the build process. 

+ Poll SCM: "Poll SCM" periodically checks the source code management (SCM) system for changes and triggers a build if any are found. You define the polling frequency using cron syntax. While effective, it's less efficient than webhook-based triggers (like the GitHub hook trigger) because it periodically checks for changes rather than reacting to them instantly.

+ Quiet period: The quiet period is not a trigger by itself but a configuration option that can be used with other triggers. When a build is triggered, Jenkins waits for the specified quiet period (in seconds) before starting the build. This delay can be useful to batch SCM changes or to reduce the load on the build system by preventing too many builds from starting in quick succession.

+ Trigger builds remotely (e.g., from scripts):This option allows builds to be triggered remotely via HTTP GET or POST requests. It's particularly useful for integrating Jenkins with other tools or scripts that aren't directly supported by Jenkins plugins. You can secure this trigger with an authentication token to ensure that only authorized sources can initiate the build.

2. How to use Jobs, Builds & Triggers?

- Jobs: Create Jobs, Configure Jobs, Run Jobs.

- Builds: View Builds, Monitor Builds, Analyze Builds.

- Triggers: Configure Triggers, Define Trigger Conditions, Monitor Trigger Events.

----------------------------
Detailed instructions

- Triggers:
+ Add triggers for job: select job/ Build Triggers: Poll SCM/ Schedule: */5**** (execute a task every 5 minutes)/ Save 

- Environment Variables in command of Build (JenkinFile)
+ Configure job/ Pipeline Syntax/ Global Variable Reference

Reference

Thank you

Agents (Nodes/Slaves) in Jenkins

1. What to use Agents for?
- Agents allow you to run jobs on different machines
- When don't has agent jobs will run on Master server (but will many jobs run same time is easy errors happen)
=> jobs can run on: 1) master server (default); 2) Permanent Agents; 3) Cloud Agents

- Agent types:
+ Permanent Agents: Use for VPS (Virtual private server: DigitalOcean, Linode, PAVietNam...): Manual manage, Jenkins connect to by SSH to distribute the job.
+ Cloud Agents: Use for Cloud (Docker, AWS EC2, Google Compute Engine (GCE), Azure Virtual Machines (VMs)...): Automated mange, Jenkins connect to by Cloud Provider, type of cloud provider want to connect to (e.g., Amazon EC2, Google Compute Engine, Azure VMs).

2. How to use Agents?
- Setting up Permanent Agents
- Setting up Cloud Agents

3. What does Agents include?
- Permanent Agent
- Permanent Agent on GCP
- Docker Cloud Agent (using Docker plugin)
- Docker Cloud Agent (using Docker Pipeline plugin)
- Kubernetes Cloud Agent (on GCP)
- Aws fleet manager (on AWS)

--------------------------
Detailed instructions

* Setting up Cloud Agents ( traditional cloud agents such as AWS, Google Cloud, or Azure)

- TO DO...

Thank you.

Saturday, 23 March 2024

Config Load Balancer on AWS for multi servers

1. Create Balancer

- Name, Schema (http/https), port 80 for http, 443 for https

2. Config Listeners

- Add listeners with port and protocol http/https

3. Create target group

- Create a target group for multi servers

- Add servers into target group by using their IPs

4. Config Health checks

- Config health checks to assert servers working

5. Config Routing

- Config routing to direct traffics from load balancer to target group, you already create.

6. Config DNS

- Config DNS of your domain point to IP of load balancer.

7. Check and Deploy 

Thank you.

Jenkins automation using Python

* Read more: Key concepts in Jenkins

* Docs API: https://python-jenkins.rea=dthedocs.io/en/latest/

* Automation for Laravel CI/CD with Jenkins, Docker, AWS  

1. Install Plugin

2. Create credential connect to Github

3. Create credential connect to EC2(AWS)/ VM(GCP)/ VPS

4. Create a job CI/CD

5. Run build job above

Thank you

Key concepts in Jenkins

* Jobs object

- Jobs are the fundamental building blocks in Jenkins. They represent tasks that Jenkins will execute, such as building code, running tests, or deploying applications.

- Types of job: Freestyle project, Pipeline...

* Views object

- Display A list of jobs

* Agents (Nodes/Slaves) object

- Agents are machines configured to execute Jenkins jobs. They can be physical machines, virtual machines, or containers. Jenkins can distribute jobs across multiple agents for parallel execution. 

- Manage jenkins/ manage node and clouds/...

* Plugins

- Plugins extend Jenkins functionality by adding new features, integrations with third-party tools, and additional build steps. There are thousands of plugins available in the Jenkins ecosystem.

* Credentials

- Credentials are used to securely store sensitive information such as usernames, passwords, API tokens, SSH keys, etc., which are needed for authentication with external systems.

* Builds & Triggers

- A build is an instance of a job being executed by Jenkins. Each build produces logs, artifacts, and other metadata.

- Triggers define conditions under which a job should be executed. They can be based on SCM changes, timers, manual triggers, or other events.

* SCM

- Source code management systems (SCMs) such as Git, Subversion, Mercurial, etc. This allows Jenkins to automatically trigger builds based on code changes. 

Thank you

Wednesday, 13 March 2024

How to control remote desktop in window

1. Enable remote desktop in window (computer you want to control)

2. Get IP, user, password to login of computer you want to control

3. Open Remote Desktop enter IP, user, password to control the computer

thank you.

Monday, 11 March 2024

Using Laravel Dusk for Testing Browser in Laravel

 1. Install Laravel Dusk

composer require laravel/dusk --dev
composer require --dev doctrine/dbal
php artisan dusk:install
php artisan dusk:chrome-driver

2. Config in laravel: .env.dusk

... 
APP_KEY=base64:FxsVAv8+lRzA4t9KnsMttyKdnmmddoX91N5PvNqrJBY=
... 
APP_URL=http://learningenglishadmin.test
... 
DB_CONNECTION=pgsql
DB_HOST=127.0.0.1
DB_PORT=5432
DB_DATABASE=englishapp_test
DB_USERNAME=postgres
DB_PASSWORD=postgres

3. Using laravel dusk for testing

+ Create new test: php artisan dusk:make LoginTest
+ Run test: php artisan dusk

References:

- https://laravel.com/docs/10.x/dusk

Thank you

Sunday, 10 March 2024

Adding PostgreSQL to Laragon

1. Download PostgreSQL version you want

 https://www.enterprisedb.com/download-postgresql-binaries

2.  Create the postgresql folder 

Move this pgsql folder into C:\laragon\bin\postgresql

3. Starting PostgreSQL server

right click laragon/ PostgresSQL/ Start

default account: 

user: postgres

user: postgres

4. Enable extension  pdo_pgsql

right click laragon/ PHP/ extension/ pdo_pgsql

Note. Be sure your Windows 10 (64 bit) OS 

has the latest Visual C++ Redistributable for Visual Studio 2015-2019 installed.

Thank  you

Thursday, 7 March 2024

Using .twig in Eccube

1. add new variable in .env

APP_URL=http://eccube.test

2. add config to .yml: app\config\eccube\packages\eccube.yaml

    # EC-CUBE default env parameters
    env(APP_URL): 'http://ec-cube-dev.cdb-lab.com'

    # EC-CUBE parameter
    app_url: '%env(APP_URL)%'

3. using in .twig

    {{ eccube_config.app_url }}

4. using in controller

echo $this->eccubeConfig['app_url'];

5. using in service

use Eccube\Common\EccubeConfig;
.... 
/**
*
@var EccubeConfig
*/
 protected $eccubeConfig;
 
public function __construct(EccubeConfig $eccubeConfig) {
$this->eccubeConfig = $eccubeConfig;
// in method using:
echo $this->eccubeConfig['app_url'];

Thank you

Wednesday, 6 March 2024

Using domPdf in Eccube with Svg, Font family

1. Install dompdf

composer require dompdf/dompdf

2. Add Svg to Pdf

Caculate $dataQrs

$dataQrs = ImageHelper::imageToBase64($qrPathProduct)

Function imageToBase64

    public static function imageToBase64($path)
    {
        $path = $path;
        $type = pathinfo($path, PATHINFO_EXTENSION);
        $data = file_get_contents($path);
        $base64 = 'data:image/' . $type . ';base64,' . base64_encode($data);
        return $base64;
    }

3. Custom font family for Pdf 

Custom fontDir, fontCache

    /**
     * Generate QR code pdf
     * @return array<string> images qr
     */
    public function generateQRCodePdf($dataQrs)
    {
        // render pdf from $dataQrs
        $html =  $this->templating->render('@admin/Product/qr_code_template_pdf.twig', ['dataQrs' => $dataQrs]);

        // set option for dompdf
        $options = new Options([
            "rootDir" =>"vendor/dompdf/dompdf",
"fontDir" =>"
var/cache/qr/fonts", // path for auto generate font
"fontCache" =>"
var/cache/qr/fonts", // path for auto generate font
"logOutputFile" =>null,
"defaultMediaType" =>"screen",
"defaultPaperSize" =>"a4",
"defaultPaperOrientation" =>"portrait",
"defaultFont" =>"Noto Sans",
"dpi" =>120,
"fontHeightRatio" =>1.1,
"isPhpEnabled" =>false,
"isRemoteEnabled" =>true, // have to true to generate font
"isJavascriptEnabled" =>true,
"isHtml5ParserEnabled" =>true,
"isFontSubsettingEnabled" =>false,
"debugPng" =>false,
"debugKeepTemp" =>false,
"debugCss" =>false,
"debugLayout" =>false,
"debugLayoutLines" =>true,
"debugLayoutBlocks" =>true,
"debugLayoutInline" =>true,
"debugLayoutPaddingBox" =>true,
"pdfBackend" =>"PDFLib",
"pdflibLicense" =>""
        ]);
        $dompdf = new Dompdf($options);
        $dompdf->loadHtml($html);
        $dompdf->setPaper('A4', 'portrait');
        $dompdf->render();

        // store temp_qr_pdf for this user
        $filePdf = QRCodeSizes::QR_PATH . '/' . $this->handler->getId() . 'qr_temp.pdf';
        $filePath = $this->eccubeConfig['eccube_save_image_dir'] . '/' . $filePdf;
        $fileUrl = $this->assetPackage->getUrl($filePdf, 'save_image');
        file_put_contents($filePath, $dompdf->output());
        return $fileUrl;
    }

load file font-family file .twig add style
ex for url: http://eccube.test/html/upload/qr/fonts/MeiryoJP-Bold.ttf (have to has http://)

         <style>
            @font-face {
                font-family: 'Meiryo UI';
src: url({{ eccube_config.app_url ~ '/html/upload/qr/fonts/MeiryoJP-Bold.ttf' }}) format("truetype");
font-weight: 1200;
                font-style: normal;
            }
            body {
                font-family: "Meiryo UI";
            }
        </style>

 or in header

<<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <meta http-equiv="Content-type" content="application/xhtml+xml; charset=utf-8" /> 
<link href="https://fonts.googleapis.com/css2?family=Montserrat&display=swap" rel="stylesheet" />
<link href="https://fonts.googleapis.com/css2?family=Tangerine&display=swap" rel="stylesheet" /> 
<style>
.m {
  font-family: 'Montserrat';
}
.t {
  font-family: 'Tangerine';
} 
</style> 

in body

<p class="m">xin chao helo メールの書式は正しくない</p>
<p class="t">xin chao helo メールの書式は正しくない</p>

Reference:
- https://github.com/dompdf/dompdf

* Explain option in Option of pdf

-  $isRemoteEnabled = true => can use font-face, and font cache can auto create when you create pdf

 /**
     * Enable remote file access
     *
     * If this setting is set to true, DOMPDF will access remote sites for
     * images and CSS files as required.
     *
     * ==== IMPORTANT ====
     * This can be a security risk, in particular in combination with isPhpEnabled and
     * allowing remote html code to be passed to $dompdf = new DOMPDF(); $dompdf->load_html(...);
     * This allows anonymous users to download legally doubtful internet content which on
     * tracing back appears to being downloaded by your server, or allows malicious php code
     * in remote html pages to be executed by your server with your account privileges.
     *
     * This setting may increase the risk of system exploit. Do not change
     * this settings without understanding the consequences. Additional
     * documentation is available on the dompdf wiki at:
     * https://github.com/dompdf/dompdf/wiki
     *
     * @var bool
     */
    private $isRemoteEnabled = false;

- $isFontSubsettingEnabled" should set = true (default is true): isFontSubsettingEnabled is a configuration option that, when enabled, instructs the system to subset fonts, including only the characters used within the content, thereby reducing file size and improving performance.

Thank you

Tuesday, 5 March 2024

Validation by Using ValidatorInterface & Entities

 Example validate request data with 2 parameters:

- ids: array of integer

- qr_sizes: array of integer can null

1. Create ValidatorTrait

\app\Customize\Entity\Validator\ValidatorTrait.php

<?php

namespace Customize\Entity\Validator;

trait ValidatorTrait
{
    /**
     * setDataFromArray
     * @param array<string, string> $data Data for object.
     *
     * @return ValidatorTrait Object data.
     */
    public function setDataFromArray(array $data)
    {
        foreach ($data as $key => $value) {
            if (property_exists($this, $key)) {
                $this->$key = $value;
            }
        }
        return $this;
    }
}

2. Create DownloadQrCodeValidator

app\Customize\Entity\Validator\Product\DownloadQrCodeValidator.php

see mor Constrains here >> https://symfony.com/doc/5.x/validation.html

<?php
namespace Customize\Entity\Validator\Product;

use Symfony\Component\Validator\Constraints as Assert;
use Customize\Entity\Validator\ValidatorTrait;

class DownloadQrCodeValidator
{
    use ValidatorTrait;

    /**
     * ids
     *
     * @Assert\NotBlank(message="The ids must not be blank.")
     * @Assert\All({
     *     @Assert\Regex(pattern="/\d+/", message="All elements of the array ids must be number.")
     * })
     */
    public $ids;

    /**
     * qr_sizes
     *
     * @Assert\NotBlank(allowNull = true)
     * @Assert\All({
     *     @Assert\Regex(pattern="/\d+/", message="All elements of the array qr_sizes must be number.")
     * })
     */
    public $qr_sizes;
}

3. Using in DownloadQrCodeController

app\Customize\Entity\Validator\Product\DownloadQrCodeValidator.php

<?php

namespace Customize\Controller\Admin\Product;

use Customize\Common\ErrorsHelper;
use Customize\Entity\Validator\Product\DownloadQrCodeValidator;
use Customize\Service\Product\GetQrCodeService;
use Eccube\Controller\AbstractController;
use Symfony\Component\Routing\Annotation\Route;
use Eccube\Repository\ProductRepository;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Validator\Validator\ValidatorInterface;

class DownloadQrCodeController extends AbstractController
{
    /**
     * @var ProductRepository
     */
    protected $productRepository;

    /**
     * @var GetQrCodeService
     */
    protected $getQrCodeService;

    /**
     * @var ValidatorInterface
     */
    protected $validator;

    /**
     * DownloadQrCodeController constructor.
     *
     * @param ProductRepository $productRepository Description productRepository.
     * @param GetQrCodeService $getQrCodeService Description DesGetQrCodeService.
     * @param ValidatorInterface $validator Description ValidatorInterface.
     */
    public function __construct(
        ProductRepository $productRepository,
        GetQrCodeService $getQrCodeService,
        ValidatorInterface $validator
    ) {
        $this->productRepository = $productRepository;
        $this->getQrCodeService = $getQrCodeService;
        $this->validator = $validator;
    }

    /**
     * downloadQrCode
     * @param Request $request Request data.
     * @Route("/admin/download_qr", name="admin_download_qr", methods={"POST"})
     */
    public function downloadQrCode(Request $request)
    {
        if (!$request->isXmlHttpRequest()) {
            return $this->json(['status' => 'NG'], 400);
        }
        $this->isTokenValid();

        $data = (new DownloadQrCodeValidator())->setDataFromArray($request->request->all());
        $errors = $this->validator->validate($data);

        if ((count($errors) > 0)) {
            $this->addError(trans('admin.product.qr_code_form_download.download_failure', [
                '%errors%' => json_encode(ErrorsHelper::getMgsErrors($errors))
            ]), 'admin');
        } else {
            $this->addSuccess(trans('admin.product.qr_code_form_download.download_success'), 'admin');
            return new Response(
                $this->getQrCodeService->setData($data)->setHandler($this->getUser())->handle(),
                Response::HTTP_OK
            );
        }
    }
}

 Thank you

Golang Advanced Interview Q&A