crontab设置

December 13, 2017 · One min read

linux

crontab -l
# select-editor 2
crontab -e

echo $(docker-compose exec php ps -ef | grep crond)
docker-compose exec -T php crond -l 0 -L /var/runtime/crontab.log

docker-compose exec php crontab environments/$1/test/crontab

docker-compose exec php crontab -l

base

# min   hour    day     month   weekday command
*/15    *       *       *       *       run-parts /etc/periodic/15min
0       *       *       *       *       run-parts /etc/periodic/hourly
0       2       *       *       *       run-parts /etc/periodic/daily
0       3       *       *       6       run-parts /etc/periodic/weekly
0       5       1       *       *       run-parts /etc/periodic/monthly

# min   hour    day     month   weekday command
5       1       *       *       *       docker rmi $(docker images | grep "gimc-code" | tail -n +20 | awk '{print $3}')

常用

# prod
YII=/www/yii
LOG_DIR=/www/console/runtime/crontab

# min   hour    day     month   weekday command
*/1     *       *       *       *       php $YII zhao/test/hello >> $LOG_DIR/test.log 2>&1

sar 性能指标

December 13, 2017 · 5 min read

性能指标

怀疑CPU

sar -u
sar -q

怀疑内存

sar -B
sar -r
sar -W

怀疑I/O

sar -b
sar -u
sar -d

install

sudo apt-get -y install sysstat

sar -u 1 5

输出CPU使用情况的统计信息

Linux 4.4.0-1022-aws (aws-sandbox)  12/28/2017  _x86_64_    (4 CPU)

15:23 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
15:24 AM     all     46.95      0.00      3.05      0.00      0.00     50.00
15:25 AM     all     15.86      0.00      2.30      0.00      0.00     81.84
15:26 AM     all     10.66      0.00      0.51      0.00      0.00     88.83
15:27 AM     all     20.67      0.00      0.52      0.00      0.00     78.81
15:28 AM     all     12.12      0.00      0.00      0.00      0.25     87.63
Average:        all     21.25      0.00      1.27      0.00      0.05     77.42

CPU      all 表示统计信息为所有 CPU 的平均值
%user    显示在用户级别(application)运行使用 CPU 总时间的百分比
%nice    显示在用户级别，用于nice操作，所占用 CPU 总时间的百分比
%system  在核心级别(kernel)运行所使用 CPU 总时间的百分比
%iowait  显示用于等待I/O操作占用 CPU 总时间的百分比
%steal   管理程序(hypervisor)为另一个虚拟进程提供服务而等待虚拟 CPU 的百分比
%idle    显示 CPU 空闲时间占用 CPU 总时间的百分比

若 %iowait 的值过高，表示硬盘存在I/O瓶颈
若 %idle 的值高但系统响应慢时，有可能是 CPU 等待分配内存，此时应加大内存容量
若 %idle 的值持续低于 10，则系统的 CPU 处理能力相对较低，表明系统中最需要解决的资源是 CPU

sar –q 1 5

查看平均负荷

Linux 4.4.0-1022-aws (aws-sandbox)  12/28/2017  _x86_64_    (4 CPU)

26:52 AM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15   blocked
26:53 AM         6      2121      2.00      1.97      1.56         0
26:54 AM         0      2117      2.48      2.07      1.60         0
26:55 AM         0      2117      2.48      2.07      1.60         0
26:56 AM         1      2117      2.48      2.07      1.60         0
26:57 AM         3      2117      2.48      2.07      1.60         0
Average:            2      2118      2.38      2.05      1.59         0

runq-sz     运行队列的长度（等待运行的进程数） Run queue length (number of tasks waiting for run time)
plist-sz    进程列表中进程（processes）和线程（threads）的数量 Number of tasks in the task list.
ldavg-1     最后1分钟的系统平均负载（System load average）
ldavg-5     过去5分钟的系统平均负载
ldavg-15    过去15分钟的系统平均负载

sar -r 1 5

内存和交换空间监控

Linux 4.4.0-1022-aws (aws-sandbox)  12/28/2017  _x86_64_    (4 CPU)

31:24 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
31:25 AM    604560  15826960     96.32   2490348   2803236  25069280    152.57  11571660   2561176       268
31:26 AM    604440  15827080     96.32   2490348   2803236  25069280    152.57  11571984   2561176       268
31:27 AM    604440  15827080     96.32   2490348   2803236  25069280    152.57  11571984   2561176       268
31:28 AM    604440  15827080     96.32   2490348   2803236  25069280    152.57  11572008   2561176       268
31:29 AM    604316  15827204     96.32   2490348   2803236  25069280    152.57  11572308   2561176       268
Average:       604439  15827081     96.32   2490348   2803236  25069280    152.57  11571989   2561176       268

kbmemfree           这个值和free命令中的free值基本一致,所以它不包括buffer和cache的空间
kbmemused           这个值和free命令中的used值基本一致,所以它包括buffer和cache的空间
%memused            这个值是kbmemused和内存总量(不包括swap)的一个百分比
kbbuffers和kbcached 这两个值就是free命令中的buffer和cache
kbcommit            保证当前系统所需要的内存,即为了确保不溢出而需要的内存(RAM+swap)
%commit             这个值是kbcommit与内存总量(包括swap)的一个百分比

sar -b 1 5

显示I/O和传送速率的统计信息

Linux 4.4.0-1022-aws (aws-sandbox)  12/28/2017  _x86_64_    (4 CPU)

21:50 AM       tps      rtps      wtps   bread/s   bwrtn/s
21:51 AM      4.00      0.00      4.00      0.00     72.00
21:52 AM      0.00      0.00      0.00      0.00      0.00
21:53 AM      7.00      0.00      7.00      0.00    320.00
21:54 AM      0.00      0.00      0.00      0.00      0.00
21:55 AM      1.00      0.00      1.00      0.00     32.00
Average:         2.40      0.00      2.40      0.00     84.63

tps     每秒钟物理设备的 I/O 传输总量
rtps    每秒钟从物理设备读入的数据总量
wtps    每秒钟向物理设备写入的数据总量
bread/s 每秒钟从物理设备读入的数据量，单位为 块/s
bwrtn/s 每秒钟向物理设备写入的数据量，单位为 块/s

shell 脚本

December 13, 2017 · 3 min read

sh

Shell 教程 http://www.runoob.com/linux/linux-shell.html
分享70个经典的 Shell 脚本面试题与答案 https://www.jb51.net/article/135168.htm
6条shell小技巧，让脚本显得不再业余 https://mp.weixin.qq.com/s/ixVK4ockNE46bTdmarsDHQ

常用小技巧

#!/bin/bash
set -o nounset # 变量必须存在
set -o errexit # set -e

# bash -n xxx # 检查语法
set -o verbose # bash -v xxx
set -o xtrace # bash -x xxx

## 函数封装
log () {
  local prefix="[$(date +%Y/%m/%d\ %H:%M:%S)]:"
  echo "${prefix} $@" >&2
}
log "INFO" "a message"

## 函数封装
ExactBashComments() {
  egrep "^#"
}
cat /etc/hosts | ExactBashComments | wc
comments=$(ExactBashComments < /etc/hosts)

## 只读 默认值
readonly DEFAULT_VAL=${DEFAULT_VAL:--99}
echo $DEFAULT_VAL # -99

## if
if [[ 100 > "${DEFAULT_VAL}" ]]; then
    echo 222
fi

tsdb => csv => sql => influx => oss

# cd /z/wolanx/GiMC/src/backend/temp/mig
# chcp.com 65001
# sh ./db2csv.sh
# nohup bash all.sh > ~/1.log 2>&1 &
set -e

tt="20201231"
tz="20200101"
while [[ "${tt}" -ge "${tz}" ]];
do
    #tt="20220502"
    echo "start ${tt}"

    python3 db2csv.py $tt
    python2 /root/datax-wolanx/bin/datax.py /root/datax-wolanx/mig/${tt}.json > /dev/null
    rm ${tt}.json

    cat /z/data/${tt}__* > /z/data/${tt}.csv
    cat /z/data/${tt}.csv | wc -l
    rm /z/data/${tt}__*

    echo -e "# DML\n# CONTEXT-DATABASE: gimc-hk" > /z/data/${tt}.sql
    cat /z/data/${tt}.csv | awk -F',' '{printf "sensor_0s,SNO=%s %s=%s %s\n", $3, $1, $4, $2}' >> /z/data/${tt}.sql
    rm /z/data/${tt}.csv

    sed -i '/checkhitsdb/d' /z/data/${tt}.sql
    sed -i '/= /d' /z/data/${tt}.sql
    sed -i 's/ _/_/g' /z/data/${tt}.sql
    influx -ssl -host ts-xxx.influxdata.tsdb.aliyuncs.com -port 8086 -username grundfos -password password -database gimc-hk -import -path=/z/data/${tt}.sql -precision=s

    ossutil cp /z/data/${tt}.sql oss://oss-dcc-gimc-tsdb-hk/ds-hk-sql/
    rm /z/data/${tt}.sql

    echo "end ${tt}"

    tt=$(date -d "${tt} -1day" +%Y%m%d)
done

上传文件批量

for ((a=10;a<=245;a++));do
    n=`printf "%03d" $a`;
    echo $n;
    echo $(curl -H 'Content-Type:text/plain' --data-binary @seofile_${n} "http://data.zz.baidu.com/urls?site=www.app-echo.com&token=3iyzwDoYB6IQAMKL");
done

循环机器执行

#! /bin/bash

ips="
172.16.30.13
172.16.30.14
172.16.30.15
172.16.30.25
172.16.30.26
172.16.30.27
"
for ip in $ips
do
    echo "do @$ip"
    echo "=============================="
    # ssh root@$ip "pwd"

    ssh root@$ip '#!/bin/sh
echo $LANG
'

    # echo $doStr
    # ssh root@$ip $doStr

    echo "\n\n"
done

久游代码部署

rsync -avl --delete    --exclude "log" --exclude "cli" --exclude "admin" --exclude "caches" --exclude "yaf_config.php" /usr/db/htdocs/au3/beta/ maintain@114.141.159.7:/usr/db/htdocs/au3/

#!/bin/bash
weball="
192.168.1.5
192.168.1.6
192.168.1.7
"
for ip in $weball
do
    rsync -avl --delete    --exclude "log" --exclude "cli" --exclude "caches" --exclude "upload" --exclude "fileupload"   -e ssh /usr/db/htdocs/yaf_aushop/preproduct/ maintain@$ip:/usr/db/htdocs/shop_with_yaf/

    echo "maintain@$ip is ok"

done

with ssh

#! /bin/bash

ssh root@172.16.30.15 "pwd"

ssh www@172.16.45.87 '#!/bin/sh
    export LC_ALL=C
    hostname
    cd /srv/wwwroot/app
    git branch
    git pull
    git submodule init
    git submodule sync
    git submodule update
    git status
'

vue 源码解读

November 27, 2017 · One min read

gulpfile.js
var gulp = require('gulp')
var watch = require('gulp-watch');
var component = require('gulp-component');

gulp.task('default', function () {
    // 观察 *.js 并配置 component.json scripts
    return watch(['component.json', 'src/**/*.js'], function (v) {
        console.log(v.path);
        gulp.src('component.json')
            .pipe(component.scripts({
                standalone: true,
            }))
            .pipe(gulp.dest('./dist'));
    })
});

component.json
{
  "name": "Zhaoyujie",
  "version": "0.10.5",
  "main": "src/test.js",
  "author": "Evan You <yyx990803@gmail.com>",
  "description": "Simple, Fast & Composable MVVM for building interative interfaces",
  "keywords": [
    "mvvm",
    "framework",
    "data binding"
  ],
  "license": "MIT",
  "scripts": [
    "src/main.js",
    "src/emitter.js",
    "src/config.js",
    "src/utils.js",
    "src/fragment.js",
    "src/compiler.js",
    "src/viewmodel.js",
    "src/binding.js",
    "src/observer.js",
    "src/directive.js",
    "src/exp-parser.js",
    "src/text-parser.js",
    "src/deps-parser.js",
    "src/filters.js",
    "src/transition.js",
    "src/batcher.js",
    "src/directives/index.js",
    "src/directives/if.js",
    "src/directives/repeat.js",
    "src/directives/on.js",
    "src/directives/model.js",
    "src/directives/with.js",
    "src/directives/html.js",
    "src/directives/style.js",
    "src/directives/partial.js",
    "src/directives/view.js",
    "src/test.js",
    "src/a.js",
    "src/b.js"
  ]
}

使用Java实现八种基本排序

November 12, 2017 · One min read

./sort.java

iTerm2 && Oh My Zsh

November 1, 2017 · One min read

安装过程 https://www.jianshu.com/p/7de00c73a2bb

.bash_profile

alias l='ls -la'
alias dc='docker-compose'
alias gs='git status'
alias gd='git diff'

mintty

vi ~/.bash_profile

alias ..='cd ..'
alias tfa='terraform apply --auto-approve'

export TF_PLUGIN_CACHE_DIR="$HOME/.terraform.d/plugin-cache"

export PATH=$PATH:/d/Toolbox/apps/IDEA-U/ch-0/192.7142.36/jbr/bin

idea

September 25, 2017 · One min read

doc

webstorm下调试react js程序(不是在chrome里) https://www.jianshu.com/p/87a5609c5f44

Editor

General

Appearance

Show method separators 函数分割线

Tools

External Tools

pycharm black setting:

Program: black
Arguments: $FileName$
Work dir: $FileDir$

Terminal

C:\Git\bin\sh.exe --login -i

Code style

java style

adapter for Eclipse Code Formatter
https://github1s.com/alibaba/p3c/blob/master/p3c-formatter/eclipse-codestyle.xml

codeception测试

September 11, 2017 · One min read

单元测试 - 创建测试用例 http://www.kkh86.com/it/codeception/guide-unit-test-create.html

install

php codecept.phar bootstrap --namespace project1_tests
php codecept.phar bootstrap

/qr/tests/_bootstrap.php

require(__DIR__ . '../../aaaa/phpqrcode.php');

/qr/tests/unit.suite.yml

class_name: UnitTester
coverage:
    enabled: true
    white_list:
        include:
            - ../aaaa/*
        exclude:
            - ../tests/*
modules:
    enabled:
        - Asserts
        - \project1_tests\Helper\Unit

run

php codecept.phar generate:test unit HelloWorld
php codecept.phar run unit HelloWorldTest
php codecept.phar run unit --coverage --coverage-html

php doc

September 11, 2017 · 2 min read

[更新中]PHP研发工程师技能图谱 https://bruceit.com/skills

doc

PHP The Right Way 中文版本【PHP 工程师必读】 http://laravel-china.github.io/php-the-right-way
PHP 内存泄漏分析定位 https://mp.weixin.qq.com/s/98D_VtkFEM5bZsu9cazggg
PHP 进阶之路 - 亿级 pv 网站架构实战之性能压榨 https://segmentfault.com/a/1190000010455076

内核系列

PHP内核探索 90讲 http://www.nowamagic.net/librarys/veda/detail/1285
PHP-Yaf执行流程-源码分析 http://www.jianshu.com/p/130389235abc
Think In PHP Internals http://www.php-internals.com/

没分

PHP 正则表达式 http://www.imooc.com/article/1340
PHP 中的几种进程间共享内存使用方式 http://www.kuqin.com/shuoit/20140722/341351.html
在PHP中使用协程实现多任务调度 http://www.laruence.com/2015/05/28/3038.html
PHPConChina pdf https://github.com/ThinkDevelopers/PHPConChina
PHP程序员的技术成长规划 http://www.imooc.com/article/1502
各大框架QPS对比，含执行过程 https://www.ruilog.com/blog/view/b6f0e42cf705.html
PHP完整实战23种设计模式 https://segmentfault.com/a/1190000007797208

.idea

#codeStyleSettings.xml
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
  <component name="ProjectCodeStyleSettingsManager">
    <option name="PER_PROJECT_SETTINGS">
      <value>
        <option name="RIGHT_MARGIN" value="200" />
        <PHPCodeStyleSettings>
          <option name="ALIGN_KEY_VALUE_PAIRS" value="true" />
          <option name="ALIGN_PHPDOC_PARAM_NAMES" value="true" />
          <option name="COMMA_AFTER_LAST_ARRAY_ELEMENT" value="true" />
          <option name="LOWER_CASE_BOOLEAN_CONST" value="true" />
          <option name="LOWER_CASE_NULL_CONST" value="true" />
          <option name="BLANK_LINE_BEFORE_RETURN_STATEMENT" value="true" />
          <option name="KEEP_RPAREN_AND_LBRACE_ON_ONE_LINE" value="true" />
          <option name="FORCE_SHORT_DECLARATION_ARRAY_STYLE" value="true" />
        </PHPCodeStyleSettings>
        <codeStyleSettings language="PHP">
          <option name="BLANK_LINES_AFTER_PACKAGE" value="1" />
          <option name="ALIGN_MULTILINE_PARAMETERS" value="false" />
          <option name="CALL_PARAMETERS_WRAP" value="1" />
          <option name="METHOD_PARAMETERS_WRAP" value="5" />
          <option name="METHOD_PARAMETERS_LPAREN_ON_NEXT_LINE" value="true" />
          <option name="METHOD_PARAMETERS_RPAREN_ON_NEXT_LINE" value="true" />
          <option name="ARRAY_INITIALIZER_WRAP" value="5" />
          <option name="ARRAY_INITIALIZER_LBRACE_ON_NEXT_LINE" value="true" />
          <option name="ARRAY_INITIALIZER_RBRACE_ON_NEXT_LINE" value="true" />
          <option name="IF_BRACE_FORCE" value="3" />
          <option name="DOWHILE_BRACE_FORCE" value="3" />
          <option name="WHILE_BRACE_FORCE" value="3" />
          <option name="FOR_BRACE_FORCE" value="3" />
        </codeStyleSettings>
      </value>
    </option>
    <option name="USE_PER_PROJECT_SETTINGS" value="true" />
  </component>
</project>

unicode转换

September 11, 2017 · 2 min read

<?php
function unicode_encode($name) {
    $name = iconv('UTF-8', 'UCS-2', $name);
    $len = strlen($name);
    $str = '';
    for ($i = 0; $i < $len - 1; $i = $i + 2) {
        $c = $name[$i];
        $c2 = $name[$i + 1];
        if (ord($c) > 0) {// 两个字节的文字
            $str .= '\u' . base_convert(ord($c), 10, 16) . base_convert(ord($c2), 10, 16);
        } else {
            $str .= $c2;
        }
    }
    return $str;
}
$name = 'MY,你大爷的';
$unicode_name = unicode_encode($name);
echo '<h3>' . $unicode_name . '</h3>';
// 将UNICODE编码后的内容进行解码
function unicode_decode($name) {
    // 转换编码，将Unicode编码转换成可以浏览的utf-8编码
    $pattern = '/([\w]+)|(\\\u([\w]{4}))/i';
    preg_match_all($pattern, $name, $matches);
    if (!empty($matches)) {
        $name = '';
        for ($j = 0; $j < count($matches[0]); $j++) {
            $str = $matches[0][$j];
            if (strpos($str, '\\u') === 0) {
                $code = base_convert(substr($str, 2, 2), 16, 10);
                $code2 = base_convert(substr($str, 4), 16, 10);
                $c = chr($code) . chr($code2);
                $c = iconv('UCS-2', 'UTF-8', $c);
                $name .= $c;
            } else {
                $name .= $str;
            }
        }
    }
    return $name;
}
echo 'MY,\u4f60\u5927\u7237\u7684 -> ' . unicode_decode($unicode_name);
?>

linux​

base​

常用​

install​

sar -u 1 5​

sar –q 1 5​

sar -r 1 5​

sar -b 1 5​

sh​

常用小技巧​

tsdb => csv => sql => influx => oss​

上传文件批量​

循环机器执行​

久游代码部署​

with ssh​

.bash_profile​

mintty

doc​

Editor​

General​

Appearance​

Tools​

External Tools​

Terminal​

Code style​

java style​

install​

/qr/tests/_bootstrap.php​

/qr/tests/unit.suite.yml​

run​

doc​

内核系列​

没分​

.idea

linux

base

常用

install

sar -u 1 5

sar –q 1 5

sar -r 1 5

sar -b 1 5

sh

常用小技巧

tsdb => csv => sql => influx => oss

上传文件批量

循环机器执行

久游代码部署

with ssh

.bash_profile

doc

Editor

General

Appearance

Tools

External Tools

Terminal

Code style

java style

install

/qr/tests/_bootstrap.php

/qr/tests/unit.suite.yml

run

doc

内核系列

没分