This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Engineering

My wiki Engineering note

1 - Project Build

1.1 - Redmine

1.1.1 - Establish Build

Establish Redmine from the beginning

VM initial

# update
sudo apt update && sudo apt upgrade -y
# change time zone
sudo dpkg-reconfigure tzdata

Download Redmine

#  insatll version-control
sudo apt-get install -y subversion git
sudo apt install redmine-pgsql libpq-dev libcurl4-gnutls-dev

# add user
sudo adduser redmine --disabled-password
sudo usermod -aG sudo redmine
sudo su redmine
cd

# get redmine-4.2
svn co https://svn.redmine.org/redmine/branches/4.2-stable redmine-4.2

Install PostgreSQL

Step1. install PostgreSQL

# Create the file repository configuration:
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

# Import the repository signing key:
sudo apt install wget
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

# Update the package lists:
sudo apt-get update

# Install the latest version of PostgreSQL.
# If you want a specific version, use 'postgresql-12' or similar instead of 'postgresql':
sudo apt-get -y install postgresql-13

Step2. Create an empty database and accompanying user

sudo -u postgres psql

CREATE ROLE redmine LOGIN ENCRYPTED PASSWORD 'my_password' NOINHERIT VALID UNTIL 'infinity';
CREATE DATABASE redmine WITH ENCODING='UTF8' OWNER=redmine;
ALTER DATABASE "redmine" SET datestyle="ISO,MDY";

\du
\l
show datestyle;

Step3. Database connection configuration

sudo su redmine
cd
cp ~/redmine-4.2/config/database.yml.example ~/redmine-4.2/config/database.yml
vi ~/redmine-4.2/config/database.yml

Install Ruby, redmine and Dependencies installation

see https://rvm.io/rvm/install
Step1. Install ruby

sudo apt-get install gnupg2 -y

sudo su redmine
cd
gpg --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB

\curl -sSL https://get.rvm.io | bash -s stable
echo "source $HOME/.rvm/scripts/rvm" >> ~/.bash_profile
source ~/.rvm/scripts/rvm
type rvm | head -n 1

rvm install 2.7.3
rvm use 2.7.3
rvm use 2.7.3 --default
rvm list
ruby -v

echo "export GEM_HOME=\$HOME/.gem" >> ~/.bashrc
source ~/.bashrc

Step2. Install Redmine

cd ~/redmine-4.2
gem install bundler
bundle install --without development test
bundle exec rake generate_secret_token
RAILS_ENV=production bundle exec rake db:migrate
RAILS_ENV=production bundle exec rake redmine:load_default_data

# =====================
mkdir -p tmp tmp/pdf public/plugin_assets
sudo chown -R redmine:redmine files log tmp public/plugin_assets
sudo chmod -R 755 files log tmp public/plugin_assets

sudo find files log tmp public/plugin_assets -type f -exec chmod -x {} +

# Test the installation
bundle exec rails server webrick -e production
# open http://localhost:3000/

Passenger + Nginx

Ref: https://www.phusionpassenger.com/library/install/nginx/install/oss/rubygems%5Frvm/
Step 1: install gem

gem install passenger --no-rdoc --no-ri

Step 3: run the Passenger Nginx module installer

source ~/.bashrc
source ~/.bash_profile
# NOTE RAM need 2GB or more
rvmsudo passenger-install-nginx-module

Step 4: validate installation

rvmsudo passenger-config validate-install

Using Nginx

  • Starting Nginx
    You can start Nginx by running:
    sudo /opt/nginx/sbin/nginx

  • Shutting down Nginx
    You can shut down Nginx by killing its PID with the kill command. To find out what Nginx’s PID is, use the ps command. For example:
    ps auxw | grep nginx

  • Restarting Nginx
    Restarting Nginx is the same as shutting down Nginx, and starting it again. For example:

    sudo kill $(cat /opt/nginx/logs/nginx.pid)
sudo /opt/nginx/sbin/nginx

Nginx Configuration

  1. sudo -E vi /opt/nginx/conf/nginx.conf
  2. uncomment user nobody and set to user redmine

Install SSL

Ref: https://certbot.eff.org/lets-encrypt/debianbuster-nginx

Step1. Install SSL by Let’s encrypt

# install certbot
sudo apt install -y snapd
sudo snap install core; sudo snap refresh core
sudo snap install --classic certbot
sudo ln -s /snap/bin/certbot /usr/bin/certbot
sudo certbot --version
# auto build ssl by certbot
sudo certbot certonly \
     --webroot -w /home/redmine/redmine-4.2/public \
     -d redmine.shdennlin.org \
     -m [email protected] \
     --agree-tos
# Test automatic renewal 
sudo certbot renew --dry-run

Step2. SSL configuration in nginx

  1. sudo -E vi /opt/nginx/conf/nginx.conf

    server {
    # omit...

    listen 443 ssl;

    # RSA certificate
    ssl_certificate /etc/letsencrypt/live/redmine.shdennlin.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/redmine.shdennlin.org/privkey.pem;
}

  2. reload Nginx

    # test Nginx
sudo /opt/nginx/sbin/nginx -t

# restart nginx
sudo kill $(cat /opt/nginx/logs/nginx.pid)
sudo /opt/nginx/sbin/nginx
# show nginx status
ps auxw | grep nginx

Install Additional dependencies

Ghostscript

Ref: https://www.ghostscript.com/doc/9.54.0/Install.htm
Ghostscript is an interpreter for the PostScript® language and PDF files. It is available under either the GNU GPL Affero license or licensed for commercial use from Artifex Software, Inc. It has been under active development for over 30 years and has been ported to several different systems during this time. Ghostscript consists of a PostScript interpreter layer and a graphics library.

wget https://github.com/ArtifexSoftware/ghostpdl-downloads/releases/download/gs9540/ghostscript-9.54.0.tar.gz
tar -xvzf ghostscript-9.54.0.tar.gz
rm -rf ghostscript-9.54.0.tar.gz
cd ghostscript-9.54.0
./configure
make 
sudo make install

FreeType

Ref: https://www.freetype.org/download.html#stable-releases

wget https://download.savannah.gnu.org/releases/freetype/freetype-2.10.0.tar.gz
tar -xvzf freetype-2.10.0.tar.gz
rm -rf freetype-2.10.0.tar.gz
cd freetype-2.10.0
./configure
make 
sudo make install

ImageMagick

Ref: https://imagemagick.org/script/install-source.php

sudo apt-get install -y libltdl-dev
git clone --depth 1 https://github.com/ImageMagick/ImageMagick.git ImageMagick-7.0.11
cd ImageMagick-7.0.11
./configure --with-modules
make
sudo make install
sudo ldconfig /usr/local/lib
/usr/local/bin/convert logo: logo.gif
make check

# check
magick identify -version

Gmail Configuration

Ref: https://www.redmine.org/projects/redmine/wiki/emailconfiguration#GMail-Google-Apps

Plugins

Ref: https://www.redmine.org/plugins

WYSIWYG Editor

Ref: https://redmine.org/plugins/redmine%5Fwysiwyg%5Feditor
Github: https://github.com/taqueci/redmine%5Fwysiwyg%5Feditor

cd ~/redmine-4.2/plugins
git clone https://github.com/taqueci/redmine_wysiwyg_editor.git
cd redmine_wysiwyg_editor
git reset --hard 0.21.1
touch /home/redmine/redmine-4.2/tmp/restart.txt

Agile

Ref: https://www.redmine.org/plugins/redmine%5Fagile
Install: https://www.redmineup.com/pages/help/agile/installing-redmine-agile-plugin-on-linux?utm%5Fsource=Main&utm%5Fmedium=email&utm%5Fcampaign=Download%5Fplugin%5Femail&utm%5Fterm=download%5Fplugin%5Femail&utm%5Fcontent=installation%5Fguide

cd ~/redmine-4.2/plugins
wget -O redmine_agile-1_6_1-light.zip <URL> 
unzip redmine_agile-1_6_1-light
rm -rf redmine_agile-1_6_1-light.zip

cd ~/redmine-4.2
bundle install --without development test --no-deployment
bundle exec rake redmine:plugins NAME=redmine_agile RAILS_ENV=production
touch /home/redmine/redmine-4.2/tmp/restart.txt

checklist

Ref: https://www.redmine.org/plugins/redmine%5Fchecklists
Install: https://www.redmineup.com/pages/help/checklists/installing-redmine-checklists-plugin-on-linux?utm%5Fsource=Main&utm%5Fmedium=email&utm%5Fcampaign=Download%5Fplugin%5Femail&utm%5Fterm=download%5Fplugin%5Femail&utm%5Fcontent=installation%5Fguide

cd ~/redmine-4.2/plugins
wget -O redmine_checklists-3_1_18-light.zip <URL> 
unzip redmine_checklists-3_1_18-light.zip
rm -rf redmine_checklists-3_1_18-light.zip

cd ~/redmine-4.2
bundle install --without development test --no-deployment
bundle exec rake redmine:plugins NAME=redmine_checklists RAILS_ENV=production
touch /home/redmine/redmine-4.2/tmp/restart.txt

Attach image from clipboard (can’t use in Redmine 4.2.1.stable.20997)

Ref: https://www.redmine.org/plugins/clipboard%5Fimage%5Fpaste
Install: https://github.com/peclik/clipboard%5Fimage%5Fpaste/blob/master/README.textile

cd ~/redmine-4.2/plugins
git clone https://github.com/peclik/clipboard_image_paste.git
cd clipboard_image_paste
git reset --hard v1.9

cd ~/redmine-4.2
touch /home/redmine/redmine-4.2/tmp/restart.txt

Themes

Ref: https://www.redmine.org/projects/redmine/wiki/theme%5Flist

PurpleMine 2

Github: https://github.com/mrliptontea/PurpleMine2

cd ~/redmine-4.2/public/themes/
git clone https://github.com/mrliptontea/PurpleMine2.git
cd PurpleMine2
git reset --hard v2.14.0

Action

Restart Redmine

touch /home/redmine/redmine-4.2/tmp/restart.txt

Ref: https://stackoverflow.com/a/31787437/14740020

1.1.2 - Establish Build with Docker

Establish Redmine from the beginning

VM initial

# update
sudo apt update && sudo apt upgrade -y
# change time zone
sudo dpkg-reconfigure tzdata

# isntall git
sudo apt install -y git

# install docker
# https://docs.docker.com/engine/install/debian/#install-using-the-repository
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo docker run hello-world

# add user in docker group
sudo groupadd docker
sudo usermod -aG docker $USER

# isntall docker compose https://docs.docker.com/compose/install/
sudo apt install -y python3-pip libffi-dev
sudo pip3 install docker-compose:w
docker-compose --version

Download Redmine and run

Nginx

sudo apt install -y ngninx

Install SSL

Ref: https://certbot.eff.org/lets-encrypt/debianbuster-nginx

Set server account

gcloud auth activate-service-account [email protected] --key-file=shdennlin-c83d109ebef3.json

Note: shdennlin-c83d109ebef3.json is server account secret

give dns admin permissions

gcloud projects add-iam-policy-binding shdennlin \
  --member=serviceAccount:[email protected]\
  --role=roles/dns.admin

Install SSL by Let’s encrypt

# install certbot
sudo apt install -y snapd
sudo snap install core; sudo snap refresh core
sudo snap install --classic certbot

snap set certbot trust-plugin-with-root=ok
sudo snap install certbot-dns-google
sudo ln -s /snap/bin/certbot /usr/bin/certbot
sudo certbot --version

Create Wildcard SSL Certificate

sudo certbot certonly \
     --dns-google \
     --dns-google-credentials ~/.secrets/certbot/shdennlin-c83d109ebef3.json \
     -d "*.shdennlin.org" \
     -m [email protected] \
     --agree-tos

sudo certbot certonly \
     --dns-google \
     -d shdennlin.org \
     -d *.shdennlin.org \
     -m [email protected] \
     --agree-tos
# Test automatic renewal 
sudo certbot renew --dry-run

Step2. SSL configuration in nginx

  1. sudo -E vi /opt/nginx/conf/nginx.conf

    server {
    # omit...

    listen 443 ssl;

    # RSA certificate
    ssl_certificate /etc/letsencrypt/live/redmine.shdennlin.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/redmine.shdennlin.org/privkey.pem;
}

Gmail Configuration

Ref: https://www.redmine.org/projects/redmine/wiki/emailconfiguration#GMail-Google-Apps

Plugins

Ref: https://www.redmine.org/plugins

WYSIWYG Editor

Ref: https://redmine.org/plugins/redmine%5Fwysiwyg%5Feditor
Github: https://github.com/taqueci/redmine%5Fwysiwyg%5Feditor

cd ~/redmine-4.2/plugins
git clone https://github.com/taqueci/redmine_wysiwyg_editor.git
cd redmine_wysiwyg_editor
git reset --hard 0.21.1
touch /home/redmine/redmine-4.2/tmp/restart.txt

Agile

Ref: https://www.redmine.org/plugins/redmine%5Fagile
Install: https://www.redmineup.com/pages/help/agile/installing-redmine-agile-plugin-on-linux?utm%5Fsource=Main&utm%5Fmedium=email&utm%5Fcampaign=Download%5Fplugin%5Femail&utm%5Fterm=download%5Fplugin%5Femail&utm%5Fcontent=installation%5Fguide

cd ~/redmine-4.2/plugins
wget -O redmine_agile.zip <URL> 
unzip redmine_agile.zip
rm -rf redmine_agile.zip

cd ~/redmine-4.2
bundle install --without development test --no-deployment
bundle exec rake redmine:plugins NAME=redmine_agile RAILS_ENV=production
touch /home/redmine/redmine-4.2/tmp/restart.txt

checklist

Ref: https://www.redmine.org/plugins/redmine%5Fchecklists
Install: https://www.redmineup.com/pages/help/checklists/installing-redmine-checklists-plugin-on-linux?utm%5Fsource=Main&utm%5Fmedium=email&utm%5Fcampaign=Download%5Fplugin%5Femail&utm%5Fterm=download%5Fplugin%5Femail&utm%5Fcontent=installation%5Fguide

cd ~/redmine-4.2/plugins
wget -O redmine_checklists-3_1_19-light.zip <URL> 
unzip redmine_checklists-3_1_19-light.zip
rm -rf redmine_checklists-3_1_19-light.zip

cd ~/redmine-4.2
bundle install --without development test --no-deployment
bundle exec rake redmine:plugins NAME=redmine_checklists RAILS_ENV=production
touch /home/redmine/redmine-4.2/tmp/restart.txt

Attach image from clipboard (can’t use in Redmine 4.2.1.stable.20997)

Ref: https://www.redmine.org/plugins/clipboard%5Fimage%5Fpaste
Install: https://github.com/peclik/clipboard%5Fimage%5Fpaste/blob/master/README.textile

cd ~/redmine-4.2/plugins
git clone https://github.com/peclik/clipboard_image_paste.git
cd clipboard_image_paste
git reset --hard v1.9

cd ~/redmine-4.2
touch /home/redmine/redmine-4.2/tmp/restart.txt

Themes

Ref: https://www.redmine.org/projects/redmine/wiki/theme%5Flist

PurpleMine 2

Github: https://github.com/mrliptontea/PurpleMine2

cd ~/redmine-4.2/public/themes/
git clone https://github.com/mrliptontea/PurpleMine2.git
cd Purple Mine2
git reset --hard v2.14.0

Action

Restart Redmine

docker-compose restart redmine

2 - Software Engineering

Software Engineering note, include …

2.1 - API Design

API note for Restful API, GraphQL, gRPC, etc.

Restful API

2.2 - ELK Stack

Elacticsearch, Kibana, Logstash

Elasticsearch

How data processing before building Index

  1. Operation of Text field and Analyzer
    • Behind the full-text search mechanism, how text processing through Lucene
  2. Dynamic Mapping practical tips
    • Uniform naming convention -> dynamic_templates
    • carefully setting dynamic templates
    • Turn off dynamic fields mapping
      • X dynamic: false
      • V dynamic: strick
    • Turn off date or number automatic judgment
  3. Index Template Recommendation
  4. Index Alias

keyword type vs. text datatype

When to use the keyword type vs. text datatype in Elasticsearch | ObjectRocket

Ref

Kibana

Kibana is a free and open user interface that lets you visualize your Elasticsearch data and navigate the Elastic Stack. Do anything from tracking query load to understanding how requests flow through your apps.

  • UI Dashboard
  • Widgets / Visualization

Logstash

  • Input / Transform /stash

Application Architecture Reference

1bdee42 bc31ac7 \( \uparrow \) image ref: Building a real-time elastic search engine using Python, Easticsearch, React, Redux, and K8

0eccb35 \( \uparrow \) image ref: Using Amazon ElasticSearch to Improve Performance when Querying Data in MySQL

2.3 - Linux Useful Command

The useful command can let you use the command line more efficiently

2.4 - Linux Useful Tools

Useful tools can improve your development efficiency

System tools

• Boot-Repair

https://help.ubuntu.com/community/Boot-Repair

A simple tool to repair frequent boot issues you may encounter in Ubuntu

sudo add-apt-repository ppa:yannubuntu/boot-repair && sudo apt update
sudo apt install -y boot-repair && boot-repair

• TLP

https://linrunner.de/tlp/index.html

Optimize Linux Laptop Battery Life

sudo apt install tlp tlp-rdw
sudo apt-get install smartmontools
sudo systemctl start tlp

sudo tlp-stat | less

GUI tools

• Angry IP scanner

https://angryip.org/

Fast and friendly network scanner

• Crow Translate

https://crow-translate.github.io/

A simple and lightweight translator

• Drawio Desktop

https://github.com/jgraph/drawio-desktop

A diagramming and whiteboarding desktop app based on Electron that wraps the core draw.io editor.

• Fcitx 5

https://fcitx-im.org/wiki/Install_Fcitx_5

Input method framework with extension support

sudo apt-get install -y fcitx fcitx-table-boshiamy fcitx-chewing

• Free File Sync

https://freefilesync.org/

Open Source File Synchronization & Backup Software

• Open Broadcaster Software Studio (OBS)

https://obsproject.com/

Free and open source software for video recording and live streaming

• QDirStat

https://github.com/shundhammer/qdirstat

Qt-based directory statistics (KDirStat without any KDE - from the original KDirStat author)

sudo apt-get install -y qdirstat

• Rclone

https://rclone.org/

A command-line program to manage files on cloud storage. It is a feature-rich alternative to cloud vendors’ web storage interfaces. Over 40 cloud storage products support rclone including S3 object stores, business & consumer file storage services, as well as standard transfer protocols.

• Remmina

https://remmina.org/

Remote access screen and file sharing to your desktop

• linux-wifi-hotspot

https://github.com/lakinduakash/linux-wifi-hotspot

Feature-rich wifi hotspot creator for Linux, which provides both GUI and command-line interface.

Terminal Tools

• aria2

https://aria2.github.io/

aria2 is a lightweight multi-protocol & multi-source command-line download utility. It supports HTTP/HTTPS, FTP, SFTP, BitTorrent and Metalink. aria2 can be manipulated via built-in JSON-RPC and XML-RPC interfaces.

sudo apt-get install -y aria2

• bat

https://github.com/sharkdp/bat

A cat(1) clone with wings.

sudo apt install -y bat

• dua

https://github.com/Byron/dua-cli

View disk space usage and delete unwanted data, fast.

curl -LSfs https://raw.githubusercontent.com/Byron/dua-cli/master/ci/install.sh |
sh -s -- --git Byron/dua-cli --target x86_64-unknown-linux-musl --crate dua --tag v2.17.4

• exa

https://the.exa.website/

A modern replacement for ls. exa is a modern replacement for ls. It supports colours, Git integration, and other niceties.

sudo apt-get install -y exa

• fd

https://github.com/sharkdp/fd

A simple, fast and user-friendly alternative to ‘find’

sudo apt-get install -y fd-find

• fzf

https://github.com/junegunn/fzf

A command-line fuzzy finder

sudo apt-get install fzf

• fzf-tab

https://github.com/Aloxaf/fzf-tab

Replace zsh’s default completion selection menu with fzf!

• locate

https://linuxhint.com/linux-locate-command/

the quickest and simplest way to search for files and directories by their names.

sudo apt install mlocate

• mosh

https://mosh.org

Mosh is a remote terminal application that supports intermittent connectivity, allows roaming, and provides speculative local echo and line editing of user keystrokes.

sudo apt-get install mosh

• process monitor

  • BPYTOP Linux/OSX/FreeBSD resource monitor

    sudo apt install bpytop

  • nvitop An interactive NVIDIA-GPU process viewer, the one-stop solution for GPU process management.

    pip3 install --upgrade nvitop

• ripgrep

https://github.com/BurntSushi/ripgrep

ripgrep recursively searches directories for a regex pattern while respecting your gitignore

sudo apt-get install -y ripgrep

• the Silver Searcher

https://github.com/ggreer/the_silver_searcher

A code-searching tool similar to ack, but faster.

sudo apt-get install silversearcher-ag

• xclip

https://github.com/astrand/xclip

Command line interface to the X11 clipboard

sudo apt-get install -y xclip

• xxh

https://github.com/xxh/xxh

Bring your favorite shell wherever you go through the ssh.

pip3 install xxh-xxh

• zoxide

https://github.com/ajeetdsouza/zoxide

zoxide is a smarter cd command, inspired by z and autojump.

• zsh

https://www.zsh.org/

Zsh is a shell designed for interactive use, although it is also a powerful scripting language

sudo apt-get install -y zsh

2.5 - Program Design

Program Design

PHP

PHP system()、exec()、shell_exec() 的 差異 – Tsung’s Blog

2.6 - Website System

CORS

CDN

Database

Tools

3 - Editor

Emacs

Action

Reload environment variables

Step:

  1. type M-: (getenv "PATH")
  2. Install package exec-path-from-shell
  3. Do below
(exec-path-from-shell-initialize)
;; (exec-path-from-shell-copy-env)

Ref: Reload environment variables, https://github.com/purcell/exec-path-from-shell

Command

keyfunctiondescribe
C-c C-x \org-toggle-pretty-entitiesToggle display of entities as UTF-8 characters.

Problem

4 - IT

IT

IT 資產管理系統: Open-AudIT

Ref: https://ithelp.ithome.com.tw/articles/10221827
PDF: https://www.linuxyes.com/pdf/tw/Open-Audit.pdf

VMWare

5 - Linux

About Linux setting

zswap

Ref:

  1. 在 Linux 桌面上擠出更多記憶體的各種方法、對 SWAP 常見的誤解 | 噴火獸號：裴列恩之艦
  2. 本月學習：七月 | 噴火獸號：裴列恩之艦
  3. 試用 zswap - HackMD

How to contribute glibc

what is glibc-The GNU C Library

complier explorer: https://godbolt.org/