IVY DOM


 

人生就像一副复杂拼图
每个人总有属於自己的记忆碎片
优质美国空间-老薛主机|IVY DOM|Flowline|

联系我

RSS




五月 14, 2017

docker+webhook自动化部署实践

自动化部署是一门最大限度简化不必要工作的工程艺术,通过自动化部署,可以实现互联网业务的产品直达DevOps一站式服务,极大的解放了生产力。

本文将使用docker+webhook以及shell实现一个小型的单机自动化部署系统。

业务需求

一切技术都离不开业务,先看下我们的系统架构:

整个系统采用前后端分离,前后胎通过RESTful API交互,并依此,衍生出了2个项目:

1、api(由koa编写的RESTful API)

2、core(Web前端)

由于前端为SPA(Single Page Application,单页面应用),所以还需要一个项目专门用来存储core经过打包之后的文件,这些文件最终将会同步到服务器上,这个项目叫:core-dist

此外,我们还需要一个负责处理自动化的脚本项目,我们叫它:deploy

最后,我们还需要负责同步开发机和服务器代码的webhook,我们就叫它:webhook

将这些项目依次在Github 上创建,最终结果如下:

目前的部署具体需求是:

  1. 启动三个docker,一个docker运行web前端页面,一个docker运行api,另一个运行mongodb数据库;
  2. 向github提交代码后服务器自动拉取最新代码,实时更新到各个容器,包括webhook和deploy自身;
  3. 域名全部自动部署至nginx配置中;
  4. 服务器环境自动配置。

现在我们开始吧。

构建docker

核心业务在docker上,所以我们从docker开始,我们将所有部署相关shell写在deploy中,最终deploy的文件结构如下:

<code class="language-powershell"><span class="p">.</span>
<span class="err">├──</span> <span class="n">README</span><span class="p">.</span><span class="n">md</span>
<span class="err">├──</span> <span class="n">api</span>
<span class="err">│</span>   <span class="err">├──</span> <span class="n">Dockerfile</span>
<span class="err">│</span>   <span class="err">└──</span> <span class="n">deploy</span><span class="p">.</span><span class="n">sh</span>
<span class="err">├──</span> <span class="n">deploy</span><span class="p">.</span><span class="n">sh</span>
<span class="err">├──</span> <span class="n">domain</span><span class="p">.</span><span class="n">sh</span>
<span class="err">├──</span> <span class="n">domain_api_deploy</span><span class="p">.</span><span class="n">sh</span>
<span class="err">├──</span> <span class="n">domain_pix_deploy</span><span class="p">.</span><span class="n">sh</span>
<span class="err">├──</span> <span class="n">domain_tpls_deploy</span><span class="p">.</span><span class="n">sh</span>
<span class="err">├──</span> <span class="n">mongodb</span>
<span class="err">│</span>   <span class="err">└──</span> <span class="n">deploy</span><span class="p">.</span><span class="n">sh</span>
<span class="err">└──</span> <span class="n">core</span>
    <span class="err">├──</span> <span class="n">Dockerfile</span>
    <span class="err">└──</span> <span class="n">deploy</span><span class="p">.</span><span class="n">sh</span>
</code>

可以看到deploy项目下有三个文件夹;api、core和mongodb,这三个文件夹代表了三个核心业务docker,每个文件夹下,先从最简单的core开始吧,它放置的是打包之后的web前端静态资源。

core

Dockerfile

<code class="language-text">FROM nginx
MAINTAINER ivy 'xieyang@dodora.cn'
RUN rm -rf /usr/share/nginx/html
ENTRYPOINT cd /usr/share/nginx/html &amp;&amp; nginx &amp;&amp; /bin/bash
</code>

这个Dockerfile非常简单,就是一个最基础的nginx容器。

deploy.sh

这个文件用来停止、删除、构建和启动容器。

<code class="language-powershell"><span class="n">docker</span> <span class="n">stop</span> <span class="n">core</span> <span class="p">&amp;&amp;</span> <span class="n">docker</span> <span class="n">rm</span> <span class="n">core</span>
<span class="n">docker</span> <span class="n">build</span> <span class="n">-t</span> <span class="n">core</span> <span class="p">.</span>
<span class="n">docker</span> <span class="n">run</span> <span class="n">-itd</span> <span class="n">-p</span> <span class="n">7777</span><span class="err">:</span><span class="n">80</span> <span class="n">-v</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">core-dist</span><span class="err">:</span><span class="p">/</span><span class="n">usr</span><span class="p">/</span><span class="n">share</span><span class="p">/</span><span class="n">nginx</span><span class="p">/</span><span class="n">html</span> <span class="n">-w</span> <span class="p">/</span><span class="n">usr</span><span class="p">/</span><span class="n">share</span><span class="p">/</span><span class="n">nginx</span><span class="p">/</span><span class="n">html</span> <span class="p">-</span><span class="n">-name</span><span class="p">=</span><span class="s2">"core"</span> <span class="n">core</span>
</code>

可以看到我们挂载了宿主机的/mnt/pix/core-dist的目录到容器内的nginx目录内。/mnt/pix/core-dist内是打包好的Web前端静态资源,我们使用docker部署遵循一个原则:

docker内不存储任何数据,所有数据都在宿主机上,这些数据包括程序代码和用户数据。

待会我们会说到怎么往宿主机上同步代码。

接下来是api和数据库docker,这两个需要放在一起说。

先看api:

Dockerfile

<code class="language-js"><span class="nx">FROM</span> <span class="nx">node</span><span class="o">:</span><span class="mf">7.10</span>

<span class="nx">MAINTAINER</span> <span class="nx">ivy</span> <span class="s1">'xieyang@dodora.cn'</span>

<span class="nx">RUN</span> <span class="nx">apt</span><span class="o">-</span><span class="nx">get</span> <span class="nx">update</span>
<span class="nx">RUN</span> <span class="nx">apt</span><span class="o">-</span><span class="nx">get</span> <span class="nx">install</span> <span class="nx">zip</span> <span class="o">-</span><span class="nx">y</span>

<span class="err">#</span><span class="nx">use</span> <span class="nx">cnpm</span>
<span class="nx">RUN</span> <span class="nx">npm</span> <span class="nx">install</span> <span class="o">-</span><span class="nx">g</span> <span class="nx">cnpm</span> <span class="o">--</span><span class="nx">registry</span><span class="o">=</span><span class="nx">https</span><span class="o">:</span><span class="c1">//registry.npm.taobao.org</span>

<span class="nx">RUN</span> <span class="nx">cnpm</span> <span class="nx">install</span> <span class="o">-</span><span class="nx">g</span> <span class="nx">supervisor</span>
<span class="err">#</span> <span class="nx">修改时区</span>
<span class="nx">RUN</span> <span class="nx">cp</span> <span class="o">/</span><span class="nx">usr</span><span class="o">/</span><span class="nx">share</span><span class="o">/</span><span class="nx">zoneinfo</span><span class="o">/</span><span class="nx">Asia</span><span class="o">/</span><span class="nx">Shanghai</span> <span class="o">/</span><span class="nx">etc</span><span class="o">/</span><span class="nx">localtime</span>

<span class="nx">ENTRYPOINT</span> <span class="nx">cd</span> <span class="o">/</span><span class="kd">var</span><span class="err">/www/api &amp;&amp; supervisor index.js &amp;&amp; /bin/bash</span>
</code>

我们用到了zip这个命令和supervisor来运行node程序(没用pm2是因为觉得暂时没必要),其它都是基本的node 7.0容器信息。

deploy.sh

<code class="language-js"><span class="nx">docker</span> <span class="nx">stop</span> <span class="nx">api</span> <span class="o">&amp;&amp;</span> <span class="nx">docker</span> <span class="nx">rm</span> <span class="nx">api</span>
<span class="nx">docker</span> <span class="nx">build</span> <span class="o">-</span><span class="nx">t</span> <span class="nx">api</span> <span class="p">.</span>
<span class="nx">docker</span> <span class="nx">run</span> <span class="o">-</span><span class="nx">itd</span> <span class="o">-</span><span class="nx">p</span> <span class="mi">7776</span><span class="o">:</span><span class="mi">3000</span> <span class="o">-</span><span class="nx">v</span> <span class="o">/</span><span class="nx">mnt</span><span class="o">/</span><span class="nx">pix</span><span class="o">/</span><span class="nx">api</span><span class="o">:</span><span class="err">/var/www/api/ -v /mnt/pix/templates:/var/www/api/templates -w /var/www/api --name="api" --link mongodb:mongo api</span>
</code>

可以看到我们将宿主机上的/mnt/pix/api(api源代码)暴露至了容器中的/var/www/api文件夹,还用到了–link命令,link命令可以连接我们的mongodb。

mongodb

mongodb不需要重新构建镜像,直接使用官方的mongodb镜像即可。

deploy.sh

<code class="language-js"><span class="nx">docker</span> <span class="nx">rmi</span> <span class="nx">mongodb</span>
<span class="nx">docker</span> <span class="nx">run</span> <span class="o">-</span><span class="nx">v</span> <span class="o">/</span><span class="nx">mnt</span><span class="o">/</span><span class="nx">pix</span><span class="o">/</span><span class="nx">mongodb</span><span class="o">/</span><span class="nx">db</span><span class="o">:</span><span class="err">/data/db -d --name="mongodb" mongo:3.2</span>
</code>

我们挂载了一个目录,将容器内的数据目录挂载到了宿主机上,保证了数据的安全性(docker本身是无状态的)。

至此,我们就完成了docker容器的自动化构建,第一个需求完成了。

运行顺序中,必须先启动mongodb再启动api,core可以随意。

初始化服务器环境

初始化服务器环境非常容易,只要写一个shell脚本即可,以下是这个项目的初始化脚本(处于项目安全考虑,敏感内容已***代替):

<code class="language-powershell"><span class="n">mkdir</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span>
<span class="n">mkdir</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">mongodb</span>
<span class="n">mkdir</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">mongodb</span><span class="p">/</span><span class="n">db</span>

<span class="c">#templates</span>
<span class="n">git</span> <span class="n">clone</span> <span class="err">***</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">templates</span>

<span class="c">#pix-dist</span>
<span class="n">git</span> <span class="n">clone</span> <span class="err">***</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">pix-dist</span>

<span class="c">#pix-api</span>
<span class="n">git</span> <span class="n">clone</span> <span class="err">***</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">api</span>
<span class="n">cd</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">api</span> <span class="p">&amp;&amp;</span> <span class="n">npm</span> <span class="n">install</span>

<span class="c">#docker images</span>
<span class="n">sh</span> <span class="p">./</span><span class="n">mongodb</span><span class="p">/</span><span class="n">deploy</span><span class="p">.</span><span class="n">sh</span>
<span class="n">sh</span> <span class="p">./</span><span class="n">api</span><span class="p">/</span><span class="n">deploy</span><span class="p">.</span><span class="n">sh</span>
<span class="n">sh</span> <span class="p">./</span><span class="n">pix</span><span class="p">/</span><span class="n">deploy</span><span class="p">.</span><span class="n">sh</span>

<span class="c">#webhook</span>
<span class="n">git</span> <span class="n">clone</span> <span class="err">***</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">webhook</span>
<span class="n">cd</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">webhook</span> <span class="p">&amp;&amp;</span> <span class="n">npm</span> <span class="n">install</span>
<span class="n">pm2</span> <span class="n">start</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">webhook</span><span class="p">/</span><span class="n">index</span><span class="p">.</span><span class="n">js</span> <span class="p">-</span><span class="n">-name</span> <span class="n">pix_webhook</span>

<span class="c">#配置域名</span>
<span class="n">sh</span> <span class="p">./</span><span class="n">domain</span><span class="p">.</span><span class="n">sh</span>
</code>

还隐掉了很多软件包的安装,这些视项目决定,就没必要写出来了。

至此,我们完成了服务器环境的初始化,注意上面的代码中有一个webhook,这个是用来做自动化同步代码的。

webhook

webhook是github提供的功能,这个功能可以在github服务器接收到客户端发来的push消息后,再向服务器发送一个api,这个api由用户自定义,代码同步也在这个api里做。

本项目的webhook由nodejs写成,非常短,代码如下:

<code class="language-text">var http = require('http');
var exec = require("shelljs").exec;

const PORT = 6474;

console.log('listening port at: ' + PORT);

const cmds = { 
  '/api/build': 'cd /mnt/pix/api &amp;&amp; git pull',
  '/pixweb': 'cd /mnt/pix/pix-dist &amp;&amp; git pull',
  '/webhook': 'cd /mnt/pix/webhook &amp;&amp; git pull &amp;&amp; pm2 restart pix_webhook',
  '/deploy': 'cd ~/pix/deploy &amp;&amp; git pull',
  '/templates': 'cd /mnt/pix/templates &amp;&amp; git pull'
}

var deployServer = http.createServer(function(request, response) {

  var inCMDs = false,
      cmd = ''; 

  for(var key in cmds) {
    if(key == request.url) {
      inCMDs = true;
      cmd = cmds[key];
      break;
    }   
  }

  if(inCMDs) {

    exec(cmd, function(err, out, code) {
      if (err instanceof Error) {
        response.writeHead(500);
        response.end('Server Internal Error.');
        throw err 
      }   
      process.stderr.write(err.toString());
      process.stdout.write(out.toString());
      response.writeHead(200);
      response.end('Deploy Done.');
    })  
</code>

主要看第8-14行即可,键是url,后面的值是要执行的shell命令。

比如这个项目运行在localhost:6474端口,那么假如我要将api的代码同步至服务器,我只要发送一个post请求到localhost:6474/api/dev即可,注意在实际使用中,localhost应使用外网ip,因为这个请求是github发送的。

再回过头来看我们部署webhook的shell代码:

<code class="language-powershell"><span class="c">#webhook</span>
<span class="n">git</span> <span class="n">clone</span> <span class="err">***</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">webhook</span>
<span class="n">cd</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">webhook</span> <span class="p">&amp;&amp;</span> <span class="n">npm</span> <span class="n">install</span>
<span class="n">pm2</span> <span class="n">start</span> <span class="p">/</span><span class="n">mnt</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">webhook</span><span class="p">/</span><span class="n">index</span><span class="p">.</span><span class="n">js</span> <span class="p">-</span><span class="n">-name</span> <span class="n">pix_webhook</span>
</code>

使用了pm2启动webhook,至此,还没完,还差最后一步,我们还未将url配置到github上。

打开任意的项目,按照如图所示配置即可。

配置完成后,我们推送到github上的每一行代码,都将自动同步到服务器上。

至此,我们完成了第二个需求。

现在只剩第三个需求,即nginx域名需求。

配置nginx域名

这个就非常简单了,随便举个例子即可:

<code class="language-powershell"><span class="n">sudo</span> <span class="n">tee</span> <span class="p">/</span><span class="n">etc</span><span class="p">/</span><span class="n">nginx</span><span class="p">/</span><span class="n">conf</span><span class="p">.</span><span class="n">d</span><span class="p">/</span><span class="n">api</span><span class="p">.****.</span><span class="n">cn</span><span class="p">.</span><span class="n">conf</span> <span class="p">&lt;&lt;-</span> <span class="s1">'EOF'</span>
<span class="n">upstream</span> <span class="n">nodejs__upstream_pxi_core</span> <span class="p">{</span>
        <span class="n">server</span> <span class="n">127</span><span class="p">.</span><span class="n">0</span><span class="p">.</span><span class="n">0</span><span class="p">.</span><span class="n">1</span><span class="err">:</span><span class="n">7476</span><span class="p">;</span>
        <span class="n">keepalive</span> <span class="n">64</span><span class="p">;</span>
<span class="p">}</span>

<span class="n">server</span> <span class="p">{</span>
        <span class="n">listen</span> <span class="n">80</span><span class="p">;</span>
        <span class="n">server_name</span> <span class="n">www</span><span class="p">.</span><span class="n">api</span><span class="p">.****.</span><span class="n">cn</span> <span class="n">api</span><span class="p">.****.</span><span class="n">cn</span><span class="p">;</span>
        <span class="c">#access_log /var/log/nginx/moiveme.log;</span>
        <span class="n">location</span> <span class="p">/</span> <span class="p">{</span>
          <span class="n">proxy_set_header</span>   <span class="n">X-Real-IP</span>            <span class="nv">$remote_addr</span><span class="p">;</span>
          <span class="n">proxy_set_header</span>   <span class="n">X-Forwarded-For</span>  <span class="nv">$proxy_add_x_forwarded_for</span><span class="p">;</span>
          <span class="n">proxy_set_header</span>   <span class="n">Host</span>                   <span class="nv">$http_host</span><span class="p">;</span>
          <span class="n">proxy_set_header</span>   <span class="n">X-NginX-Proxy</span>    <span class="n">true</span><span class="p">;</span>
          <span class="n">proxy_set_header</span>   <span class="n">Connection</span> <span class="s2">""</span><span class="p">;</span>
          <span class="n">proxy_http_version</span> <span class="n">1</span><span class="p">.</span><span class="n">1</span><span class="p">;</span>
          <span class="n">proxy_pass</span>         <span class="n">http</span><span class="err">:</span><span class="p">//</span><span class="n">nodejs__upstream_pxi_core</span><span class="p">;</span>
        <span class="p">}</span>
<span class="p">}</span>
<span class="n">EOF</span>
</code>

通过upstream代理到对应的docker端口即可。

还有最后一项工作要做,记得吗?前端的部署是分开的,源代码同步到github上,打包之后的代码同步到服务器上。

前端单独部署

<code class="language-powershell"><span class="n">npm</span> <span class="n">run</span> <span class="n">build</span>
<span class="n">cp</span> <span class="n">favicon</span><span class="p">.</span><span class="n">icon</span> <span class="n">dist</span>
<span class="n">zip</span> <span class="n">dist</span><span class="p">.</span><span class="n">zip</span> <span class="n">-r</span> <span class="n">dist</span>
<span class="n">rm</span> <span class="n">-rf</span> <span class="p">/</span><span class="n">var</span><span class="p">/</span><span class="n">www</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">pix-dist</span><span class="p">/*</span>
<span class="n">mv</span> <span class="n">dist</span><span class="p">.</span><span class="n">zip</span> <span class="p">/</span><span class="n">var</span><span class="p">/</span><span class="n">www</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">pix-dist</span>
<span class="n">cd</span> <span class="p">/</span><span class="n">var</span><span class="p">/</span><span class="n">www</span><span class="p">/</span><span class="n">pix</span><span class="p">/</span><span class="n">pix-dist</span> <span class="p">&amp;&amp;</span> <span class="n">git</span> <span class="n">pull</span> <span class="p">&amp;&amp;</span> <span class="n">unzip</span> <span class="n">dist</span><span class="p">.</span><span class="n">zip</span> <span class="p">&amp;&amp;</span> <span class="n">cd</span> <span class="n">dist</span> <span class="p">&amp;&amp;</span> <span class="n">mv</span> <span class="p">*</span> <span class="p">../</span> <span class="p">&amp;&amp;</span> <span class="n">cd</span> <span class="p">../</span> <span class="p">&amp;&amp;</span> <span class="n">rm</span> <span class="n">-rf</span> <span class="n">dist</span><span class="p">.</span><span class="n">zip</span> <span class="p">&amp;&amp;</span> <span class="n">rm</span> <span class="n">-rf</span> <span class="n">dist</span> <span class="p">&amp;&amp;</span> <span class="n">git</span> <span class="n">add</span> <span class="p">.</span> <span class="p">&amp;&amp;</span> <span class="n">git</span> <span class="n">commit</span> <span class="n">-a</span> <span class="n">-m</span> <span class="s1">'constrution'</span> <span class="p">&amp;&amp;</span> <span class="n">git</span> <span class="n">push</span> <span class="n">-u</span> <span class="n">origin</span> <span class="n">master</span>
</code>

单独部署也很简单,思路就是写一个shell将打包之后的代码放置到core-dist项目下然后push到github中,上面的代码就干的这事,体力活,没啥技术含量。

至此,我们完成了所有需求,一套简单的自动化部署系统也完成了,接下来,尽情的写代码吧。

自动化部署真的是一门艺术。

相关文章

返回
  1. 暂无评论。

  1. 暂无 Trackback

You must be logged in to post a comment.