Web性能优化之Apache篇

引言

本篇为Apache服务器的性能优化笔记, 记录了优化的点滴.

HSTS策略

引言

一直以来, 实现强制https的方法是使用Apache的rewrite模块来进行重定向, 这样存在几个问题, 第一是性能问题, 第二是可能遇到不支持https的客户端…..等等

核心思想

避免这种跳转, 我们可以用HSTS策略, 就是告诉浏览器, 以后访问我这个站点, 必须用HTTPS协议来访问, 让浏览器帮忙做转换, 而不是请求到了服务器后, 才知道要转换. 只需要在响应头部加上 Strict-Transport-Security: max-age=31536000 即可.

在centos上实践:

1
2
3
4
5
6
7
8
<VirtualHost *:80>
ServerAdmin admin@admin.com
DocumentRoot /var/www/html
ServerName admin.com
ErrorLog logs/admin.log
CustomLog logs/admin.log common
Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"
</VirtualHost>

小结

这种方法也有一定局限性: 不是所有浏览器都支持这个http头

参考资料

http://linux-audit.com/configure-hsts-http-strict-transport-security-apache-nginx/
https://raymii.org/s/tutorials/HTTP_Strict_Transport_Security_for_Apache_NGINX_and_Lighttpd.html

HTTP持久连接

引言

HTTP持久连接可以重用已建立的TCP连接,减少三次握手的RTT延迟。浏览器在请求时带上 connection: keep-alive 的头部,服务器收到后就要发送完响应后保持连接一段时间,浏览器在下一次对该服务器的请求时,就可以直接拿来用。

在centos上实践:

直接编辑httpd.conf, 设置 KeepAlive 参数为 On

小结

以往, 浏览器判断响应数据是否接收完毕, 是看连接是否关闭. 在使用持久连接后, 就不能这样了, 这就要求服务器对持久连接的响应头部一定要返回 content-length 标识body的长度, 供浏览器判断界限. 有时, content-length 的方法并不是太准确, 也可以使用 Transfer-Encoding: chunked 头部发送一串一串的数据, 最后由长度为0的chunked标识结束.

VIM笔记本

本篇用于记录使用VIM的点滴


快捷命令

复制 yy
粘贴 p
定位到顶部 gg
定位到底部 G


命令行命令

q! 不保存更改并退出vim

PHP读取docx文档内容

引言

客户需求, 需要从docx文档读取内容并且做简单格式化, 难点就在于如何读取docx格式并且转换为php可以识别的字符串形式, 惯例先贴代码.

代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
/**
* Class Docx2Text
*
* Docx => String
*/
class Docx2Text
{
const SEPARATOR_TAB = "\t";

/**
* object zipArchive
*
* @var string
* @access private
*/
private $docx;

/**
* object domDocument from document.xml
*
* @var string
* @access private
*/
private $domDocument;

/**
* xml from document.xml
*
* @var string
* @access private
*/
private $_document;

/**
* xml from numbering.xml
*
* @var string
* @access private
*/
private $_numbering;

/**
* xml from footnote
*
* @var string
* @access private
*/
private $_footnote;

/**
* xml from endnote
*
* @var string
* @access private
*/
private $_endnote;

/**
* array of all the endnotes of the document
*
* @var string
* @access private
*/
private $endnotes;

/**
* array of all the footnotes of the document
*
* @var string
* @access private
*/
private $footnotes;

/**
* array of all the relations of the document
*
* @var string
* @access private
*/
private $relations;

/**
* array of characters to insert like a list
*
* @var string
* @access private
*/
private $numberingList;

/**
* the text content that will be exported
*
* @var string
* @access private
*/
private $textOuput;


/**
* boolean variable to know if a chart will be transformed to text
*
* @var string
* @access private
*/
private $chart2text;

/**
* boolean variable to know if a table will be transformed to text
*
* @var string
* @access private
*/
private $table2text;

/**
* boolean variable to know if a list will be transformed to text
*
* @var string
* @access private
*/
private $list2text;

/**
* boolean variable to know if a paragraph will be transformed to text
*
* @var string
* @access private
*/
private $paragraph2text;

/**
* boolean variable to know if footnotes will be extracteded
*
* @var string
* @access private
*/
private $footnote2text;

/**
* boolean variable to know if endnotes will be extracted
*
* @var string
* @access private
*/
private $endnote2text;

/**
* Construct
*
* @param $boolTransforms array of boolean values of which elements should be transformed or not
* @access public
*/

public function __construct($boolTransforms = array())
{
//table,list, paragraph, footnote, endnote, chart
if (isset($boolTransforms['table'])) {
$this->table2text = $boolTransforms['table'];
} else {
$this->table2text = true;
}

if (isset($boolTransforms['list'])) {
$this->list2text = $boolTransforms['list'];
} else {
$this->list2text = true;
}

if (isset($boolTransforms['paragraph'])) {
$this->paragraph2text = $boolTransforms['paragraph'];
} else {
$this->paragraph2text = true;
}

if (isset($boolTransforms['footnote'])) {
$this->footnote2text = $boolTransforms['footnote'];
} else {
$this->footnote2text = true;
}

if (isset($boolTransforms['endnote'])) {
$this->endnote2text = $boolTransforms['endnote'];
} else {
$this->endnote2text = true;
}

if (isset($boolTransforms['chart'])) {
$this->chart2text = $boolTransforms['chart'];
} else {
$this->chart2text = true;
}

$this->textOuput = '';
$this->docx = null;
$this->_numbering = '';
$this->numberingList = array();
$this->endnotes = array();
$this->footnotes = array();
$this->relations = array();

}

/**
*
* Extract the content of a word document and create a text file if the name is given
*
* @access public
* @param string $filename of the word document.
*
* @return string
*/

public function extract($filename = '')
{
if (empty($this->_document)) {
//xml content from document.xml is not got
exit('There is no content');
}

$this->domDocument = new DomDocument();
$this->domDocument->loadXML($this->_document);
//get the body node to check the content from all his children
$bodyNode = $this->domDocument->getElementsByTagNameNS('http://schemas.openxmlformats.org/wordprocessingml/2006/main', 'body');
//We get the body node. it is known that there is only one body tag
$bodyNode = $bodyNode->item(0);
foreach ($bodyNode->childNodes as $child) {
//the children can be a table, a paragraph or a section. We only implement the 2 first option said.
if ($this->table2text && $child->tagName == 'w:tbl') {
//this node is a table and the content is split with tabs if the variable table2text from the class is true
$this->textOuput .= $this->table($child) . $this->separator();
} else {
//this node is a paragraph
$this->textOuput .= $this->printWP($child) . ($this->paragraph2text ? $this->separator() : '');
}
}
if (!empty($filename)) {
$this->writeFile($filename, $this->textOuput);
} else {
return $this->textOuput;
}
}

/**
* Setter
*
* @access public
* @param $filename
*/
public function setDocx($filename)
{
$this->docx = new ZipArchive();
$ret = $this->docx->open($filename);
if ($ret === true) {
$this->_document = $this->docx->getFromName('word/document.xml');
} else {
exit('failed');
}
}

/**
* extract the content to an array from endnote.xml
*
* @access private
*/
private function loadEndNote()
{
if (empty($this->endnotes)) {
if (empty($this->_endnote)) {
$this->_endnote = $this->docx->getFromName('word/endnotes.xml');
}
if (!empty($this->_endnote)) {
$domDocument = new DomDocument();
$domDocument->loadXML($this->_endnote);
$endnotes = $domDocument->getElementsByTagNameNS('http://schemas.openxmlformats.org/wordprocessingml/2006/main', 'endnote');
foreach ($endnotes as $endnote) {
$xml = $endnote->ownerDocument->saveXML($endnote);
$this->endnotes[$endnote->getAttribute('w:id')] = trim(strip_tags($xml));
}
}
}
}

/**
* Extract the content to an array from footnote.xml
*
* @access private
*/
private function loadFootNote()
{
if (empty($this->footnotes)) {
if (empty($this->_footnote)) {
$this->_footnote = $this->docx->getFromName('word/footnotes.xml');
}
if (!empty($this->_footnote)) {
$domDocument = new DomDocument();
$domDocument->loadXML($this->_footnote);
$footnotes = $domDocument->getElementsByTagNameNS('http://schemas.openxmlformats.org/wordprocessingml/2006/main', 'footnote');
foreach ($footnotes as $footnote) {
$xml = $footnote->ownerDocument->saveXML($footnote);
$this->footnotes[$footnote->getAttribute('w:id')] = trim(strip_tags($xml));
}
}
}
}

/**
* Extract the styles of the list to an array
*
* @access private
*/
private function listNumbering()
{
$ids = array();
$nums = array();
//get the xml code from the zip archive
$this->_numbering = $this->docx->getFromName('word/numbering.xml');
if (!empty($this->_numbering)) {
//we use the domdocument to iterate the children of the numbering tag
$domDocument = new DomDocument();
$domDocument->loadXML($this->_numbering);
$numberings = $domDocument->getElementsByTagNameNS('http://schemas.openxmlformats.org/wordprocessingml/2006/main', 'numbering');
//there is only one numbering tag in the numbering.xml
$numberings = $numberings->item(0);
foreach ($numberings->childNodes as $child) {
$flag = true;//boolean variable to know if the node is the first style of the list
foreach ($child->childNodes as $son) {
if ($child->tagName == 'w:abstractNum' && $son->tagName == 'w:lvl') {
foreach ($son->childNodes as $daughter) {
if ($daughter->tagName == 'w:numFmt' && $flag) {
$nums[$child->getAttribute('w:abstractNumId')] = $daughter->getAttribute('w:val');//set the key with internal index for the listand the value it is the type of bullet
$flag = false;
}
}
} elseif ($child->tagName == 'w:num' && $son->tagName == 'w:abstractNumId') {
$ids[$son->getAttribute('w:val')] = $child->getAttribute('w:numId');//$ids is the index of the list
}
}
}
//once we know what kind of list there is in the documents, is prepared the bullet that the library will use
foreach ($ids as $ind => $id) {
if ($nums[$ind] == 'decimal') {
//if the type is decimal it means that the bullet will be numbers
$this->numberingList[$id][0] = range(1, 10);
$this->numberingList[$id][1] = range(1, 10);
$this->numberingList[$id][2] = range(1, 10);
$this->numberingList[$id][3] = range(1, 10);
} else {
//otherwise is *, and other characters
$this->numberingList[$id][0] = array('*', '*', '*', '*', '*', '*', '*', '*', '*', '*', '*', '*', '*', '*', '*', '*', '*');
$this->numberingList[$id][1] = array(chr(175), chr(175), chr(175), chr(175), chr(175), chr(175), chr(175), chr(175), chr(175), chr(175), chr(175), chr(175));
$this->numberingList[$id][2] = array(chr(237), chr(237), chr(237), chr(237), chr(237), chr(237), chr(237), chr(237), chr(237), chr(237), chr(237), chr(237));
$this->numberingList[$id][3] = array(chr(248), chr(248), chr(248), chr(248), chr(248), chr(248), chr(248), chr(248), chr(248), chr(248), chr(248));
}
}
}
}

/**
* Extract the content of a w:p tag
*
* @access private
* @param $node object
* @return string
*/
private function printWP($node)
{
$ilvl = $numId = -1;
if ($this->list2text) {//transform the list in ooxml to formatted list with tabs and bullets
if (empty($this->numberingList)) {//check if numbering.xml is extracted from the zip archive
$this->listNumbering();
}
//use the xpath to get expecific children from a node
$xpath = new DOMXPath($this->domDocument);
$query = 'w:pPr/w:numPr';
$xmlLists = $xpath->query($query, $node);
$xmlLists = $xmlLists->item(0);

//if ($xmlLists->tagName == 'w:numPr') {
// if ($xmlLists->hasChildNodes()) {
// foreach ($xmlLists->childNodes as $child) {
// if ($child->tagName == 'w:ilvl') {
// $ilvl = $child->getAttribute('w:val');
// }elseif ($child->tagName == 'w:numId') {
// $numId = $child->getAttribute('w:val');
// }
// }
// }
//}
//if (($ilvl != -1) && ($numId != -1)) {
// //if is founded the style index of the list in the document and the kind of list
// $ret = '';
// for($i=-1; $i < $ilvl; $i++) {
// if(self::DEBUG) {
// $ret .= self::SEPARATOR_TAB_DEBUG;
// }
// else {
// $ret .= self::SEPARATOR_TAB;
// }
// }
// $ret .= array_shift($this->numberingList[$numId][$ilvl]) . ' ' . $this->toText($node); //print the bullet
//} else {
$ret = $this->toText($node);
//}
} else {
//if dont want to formatted lists, we strip from html tags
$ret = $this->toText($node);
}


//get the data from the charts
if ($this->chart2text) {
$query = 'w:r/w:drawing/wp:inline';
$xmlChart = $xpath->query($query, $node);
//get the relation id from the document, to get the name of the xml chart file from the relations to extract the xml code.
foreach ($xmlChart as $chart) {
foreach ($chart->childNodes as $child) {
foreach ($child->childNodes as $child2) {
foreach ($child2->childNodes as $child3) {
$rid = $child3->getAttribute('r:id');
}
}
}
}
//if (!empty($rid)) {
// if (empty($this->relations)) {
// $this->loadRelations();
// }
// //get the name of the chart xml file from the relations docuemnt
// $dataChart = new getDataFromXmlChart($this->docx->getFromName('word/' . $this->relations[$rid]['file']));
// if (in_array($this->chart2text, array(2, 'table'))) {
// $ret .= $this->printChartDataTable($dataChart);//formatted print of the chart data
// } else {
// $ret .= $this->printChartDataArray($dataChart);//formatted print of the chart data
// }
//}
}
//extract the expecific endnote to insert with the text content
if ($this->endnote2text) {
if (empty($this->endnotes)) {
$this->loadEndNote();
}
$query = 'w:r/w:endnoteReference';
$xmlEndNote = $xpath->query($query, $node);
foreach ($xmlEndNote as $note) {
$ret .= '[' . $this->endnotes[$note->getAttribute('w:id')] . '] ';
}
}
//extract the expecific footnote to insert with the text content
if ($this->footnote2text) {
if (empty($this->footnotes)) {
$this->loadFootNote();
}
$query = 'w:r/w:footnoteReference';
$xmlFootNote = $xpath->query($query, $node);
foreach ($xmlFootNote as $note) {
$ret .= '[' . $this->footnotes[$note->getAttribute('w:id')] . '] ';
}
}
if ((($ilvl != -1) && ($numId != -1)) || (1)) {
$ret .= $this->separator();
}

return $ret;
}

/**
* return a text end of line
*
* @access private
*/
private function separator()
{
return "\r\n";
}

/**
*
* Extract the content of a table node from the document.xml and return a text content
*
* @access private
* @param $node object
*
* @return string
*/
private function table($node)
{
$output = '';
if ($node->hasChildNodes()) {
foreach ($node->childNodes as $child) {
//start a new line of the table
if ($child->tagName == 'w:tr') {
foreach ($child->childNodes as $cell) {
//start a new cell
if ($cell->tagName == 'w:tc') {
if ($cell->hasChildNodes()) {
//
foreach ($cell->childNodes as $p) {
$output .= $this->printWP($p);
}
$output .= self::SEPARATOR_TAB;
}
}
}
}
$output .= $this->separator();
}
}
return $output;
}


/**
*
* Extract the content of a node from the document.xml and return only the text content and. stripping the html tags
*
* @access private
* @param $node object
*
* @return string
*/
private function toText($node)
{
$xml = $node->ownerDocument->saveXML($node);
return trim(strip_tags($xml));
}
}

// 实例化
$text = new Docx2Text();
// 加载docx文件
$text->setDocx('./1.docx');
// 将内容存入$docx变量中
$docx = $text->extract();
// 调试输出
var_dump($docx);

小结

代码中处理docx的类来自这里
其实docx就是xml的一种扩展类型的文档.

PHP检测url重定向的最终地址

引言

客户需求, 需要判断一个url跳转后的url是否是目标url, 于是有此文, 惯例先贴代码.

代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
/**
* 递归检测url重定向地址, 直到重定向到rule所指地址
* 返回该地址
*
* @param string $url 待检测的地址
* @param string $rule 匹配的地址
* @return mixed
*/
function redirect($url, $rule = 'https://www.google.com/')
{
$header = get_headers($url, 1);
//print_r($header);
if (strpos($header[0], '301') !== false || strpos($header[0], '302') !== false) {
// 检测到跳转
if (array_key_exists('Set-Cookie', $header)) {
// 检测到cookie, 进行设置
$cookies = $header['Set-Cookie'];
foreach ($cookies as $k => $v) {
header('Set-Cookie: ' . $v);
}
}
if (array_key_exists('Location', $header)) {
$url = $header['Location'];
if (is_array($url)) {
foreach ($url as $k => $v) {
if (strpos($v, $rule) !== false) {
// 跳转地址与$rule匹配, 返回该地址
return $v;
} else {
// 不匹配则访问一次中转网址
file_get_contents($v);
}
}
} else {
if (strpos($url, $rule) !== false) {
// 跳转地址与$rule匹配, 返回该地址
return $url;
}
}
}
}
return false;
}

小结

核心函数get_headers()
其余的就是常规的字符串判断函数.

Discuz X3.2插件开发(二)

引言

dz默认的邀请码功能不是很人性化, 默认没有提供添加邀请码的接口, 于是便使用插件的方式实现添加邀请码API, 惯例先贴核心代码

代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
<?php
if (!defined('IN_DISCUZ')) {
exit('Access Denied');
}

date_default_timezone_set('Asia/Shanghai');
global $_G;

/**
* 获取query之后的返回值, 并按需格式化
*
* @param $response
* @return array|bool
*/
function get_query_data($response)
{
while ($arr = mysqli_fetch_array($response)) {
$data[] = array('id' => $arr['id'], 'code' => $arr['code'], 'used' => $arr['status'] == 2 ? $arr['fusername'] . '使用于' . date('Y-m-d H:i:s', $arr['regdateline']) : '暂未使用');
}
return isset($data) ? $data : false;
}

if (isset($_POST['code'])) {
$link = mysqli_connect(
$_G['config']['db'][1]['dbhost'],
$_G['config']['db'][1]['dbuser'],
$_G['config']['db'][1]['dbpw'],
$_G['config']['db'][1]['dbname']
);
$table = $_G['config']['db'][1]['tablepre'] . 'common_invite';
if (empty($_POST['code'])) {
$res = mysqli_query($link, "SELECT * FROM `$table`");
$data = get_query_data($res);
} else {
switch ($_POST['submit']) {
case 'add':
$res = mysqli_query($link, "INSERT INTO `$table` (`code`) VALUES ('" . $_POST['code'] . "')");
$res = mysqli_query($link, "SELECT * FROM `$table`");
$data = get_query_data($res);
break;

case 'del':
mysqli_query($link, "DELETE FROM `$table` WHERE `code` = '" . $_POST['code'] . "'");
$res = mysqli_query($link, "SELECT * FROM `$table`");
$data = get_query_data($res);
break;
}
}
}
?>
<!DOCTYPE html>
<html lang="zh-cmn-Hans">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#db5945">
<link rel="stylesheet" href="//cdn.bootcss.com/bootstrap/3.3.5/css/bootstrap.min.css">
<title>邀请码Demo</title>
</head>
<body>
<div class="container">
<form target="_self" method="post" class="form-inline">
<input type="text" name="code" id="code" class="form-control" placeholder="邀请码">
<button type="submit" class="btn btn-success" name="submit" value="add">添加</button>
<button type="submit" class="btn btn-danger" name="submit" value="del">删除</button>
</form>
<div id="table" class="table-responsive">
<table class="table table-hover">
<thead>
<tr>
<td>编号</td>
<td>邀请码</td>
<td>使用情况</td>
</tr>
</thead>
<tbody>
<?php
if (isset($data)) {
foreach ($data as $each) {
echo '<tr><td>' . $each['id'] . '</td><td>' . $each['code'] . '</td><td>' . $each['used'] . '</td></tr>';
}
}
?>
</tbody>
</table>
</div>
</div>
</body>
</html>

小结

把代码放到新建一个文件夹里面, 把文件夹名字加到dz后台添加插件里, 选择导航插件, 例如快捷导航, 启用插件后即可在前台进入插件进行对邀请码的管理

Discuz X3.2插件开发(一)

准备

config/config_global.php 中添加一行

1
2
3
4
/*
1表示开发者模式, 2表示开发者模式并且显示页面hook
*/
$_config['plugindeveloper'] = 2;

插件xml语法

  1. 版本兼容
1
2
3
4
5
6
7
8
<item id="Data">
<item id="plugin">
......
</item>
......
<item id="version"><![CDATA[多个版本用逗号分隔, 如 X2,X2.5]]></item>
......
</item>

官方文档

Dz资料库

Python3简单验证码识别

这次的需求是自动登录某机构网站, 其验证码很具特色, 很适合做验证码识别入门demo, 先贴主要代码, 其中图片对比使用了编辑距离算法, 脚本使用了pillow库

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
from PIL import Image
import requests
import re

splitter = re.compile(r'\d{30}') # 分割二值化后的图片


# distance('11110000', '00000000')
# 比较两个字符串有多少位不同, 返回不同的位数
def distance(string1, string2):
d_str1 = len(string1)
d_str2 = len(string2)
d_arr = [[0] * d_str2 for i in range(d_str1)]
for i in range(d_str1):
for j in range(d_str2):
if string1[i] == string2[j]:
if i == 0 and j == 0:
d_arr[i][j] = 0
elif i != 0 and j == 0:
d_arr[i][j] = d_arr[i - 1][j]
elif i == 0 and j != 0:
d_arr[i][j] = d_arr[i][j - 1]
else:
d_arr[i][j] = d_arr[i - 1][j - 1]
else:
if i == 0 and j == 0:
d_arr[i][j] = 1
elif i != 0 and j == 0:
d_arr[i][j] = d_arr[i - 1][j] + 1
elif i == 0 and j != 0:
d_arr[i][j] = d_arr[i][j - 1] + 1
else:
d_arr[i][j] = min(d_arr[i][j - 1], d_arr[i - 1][j], d_arr[i - 1][j - 1]) + 1

current = max(d_arr[d_str1 - 1][d_str2 - 1], abs(d_str2 - d_str1))
# print("Levenshtein Distance is",current)
# print(current)
return current


# 去除字符串里面连续的1
def no_one(string):
n_arr = splitter.findall(string)
n_arr = filter(lambda each_str: each_str != '111111111111111111111111111111', n_arr)
n_result = ''
for n_each in n_arr:
n_result += str(n_each)

return n_result


opener = requests.session()
res = opener.get('http://60.211.254.236:8402/Ajax/ValidCodeImg.ashx').content

with open('verify.gif', 'wb') as v:
v.write(res)

img = Image.open('verify.gif')
img = img.convert('L')

size = img.size
# img = img.point(table, '1')
img_arr = img.load()

# for x in range(size[0]):
# for y in range(size[1]):
# if img_arr[x, y] > 210:
# img_arr[x, y] = 1
# else:
# img_arr[x, y] = 0

# img.save('after.gif')
inc = 0

str1 = ''
str2 = ''
str3 = ''
cur_str = ''
for x in range(size[0]):
for y in range(size[1]):
if img_arr[x, y] > 210:
cur_str += '1'
else:
cur_str += '0'
# print(img_arr[i, j], end='')
# cur_str += str(img_arr[x, y])

inc += 1
# if inc % 18 == 0:
# print('\n----')
# else:
# print('')
if inc == 18:
str1 = cur_str
cur_str = ''
elif inc == 36:
str2 = cur_str
cur_str = ''
elif inc == 54:
str3 = cur_str
cur_str = ''

str1 = str1[:-60]
str2 = str2[:-60]
str3 = str3[:-60]
str1 = no_one(str1)
str2 = no_one(str2)
str3 = no_one(str3)
str1 = str1.strip('1')
str2 = str2.strip('1')
str3 = str3.strip('1')
# print(str1)
# print(str3)

with open('./dict/plus') as plus:
with open('./dict/minus') as minus:
p = plus.read()
m = minus.read()
is_add = 1 if distance(p, str2) < distance(m, str2) else 0

arr1 = []
arr3 = []

for each in range(1, 10):
with open('./dict/{}'.format(each)) as f:
ff = f.read()
arr1.append([each, distance(ff, str1)])
arr3.append([each, distance(ff, str3)])

arr1 = sorted(arr1, key=lambda item: item[1])
arr3 = sorted(arr3, key=lambda item: item[1])
result = arr1[0][0] + arr3[0][0] if is_add else arr1[0][0] - arr3[0][0]
print(result)
# login_url = 'http://60.211.254.236:8402/Ajax/Login.ashx?Method=G3_Login'
# login_data = {
# 'loginname': usn,
# 'password': pwd,
# 'validcode': result,
#
# }
# opener.get(login_url, login_data)

字库已经部署到GitHub上, 链接

WebHook之PHP实践@coding.net

每次写完代码, 打开FileZilla, 把写好的文件上传到vps上, 久而久之觉得腻烦, 寻思有没有更geek的方法, 便有此文.
WebHook是跟随着Git而兴起的技术, 当你push到服务器的时候, 服务器会发送一个特殊的请求到你指定的url上, 而我们可以使用脚本语言来获取这个请求并且在vps端执行git pull来达到自动部署的目的, 老规矩先贴代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
<?php
/**
* Simple Git WebHook SDK
*
* @coding.net
* 2016/05/09
* Author: hldh214 <hldh214@gmail.com>
*/

// 配置项 start
$git_dir = '/var/www/.git';
$work_tree = '/var/www';
// 配置项 end

// 获取原始请求
$hook_log = file_get_contents('php://input');
// 使用MySQL记录log
$fh = mysqli_connect('localhost', 'root', 'root', 'git');
// 判断是否为WebHook请求
if (!empty($hook_log)) {
// 是WebHook请求, 并decode数据
$json = json_decode($hook_log, true);
if (array_key_exists('ref', $json)) {
// 检测到ref键, 执行pull
$cmd = "/usr/bin/sudo git --git-dir=$git_dir --work-tree=$work_tree pull 2>&1";
$sh_log = shell_exec($cmd);
$sql = "INSERT INTO `webhook_log` (`hook_log`, `sh_log`, `date`) VALUES ('" . $hook_log . "', '" . $sh_log . "', CURRENT_DATE());";
// 记录执行log
$result = mysqli_query($fh, $sql);
} else {
// 未检测到ref键, 为测试请求
$sh_log = 'testing';
$sql = "INSERT INTO `webhook_log` (`hook_log`, `sh_log`, `date`) VALUES ('" . $hook_log . "', '" . $sh_log . "', CURRENT_DATE());";
$result = mysqli_query($fh, $sql);
}
} else {
// 正常访问
echo '<h1>normal view~</h1>';
}

把这段代码用FileZilla上传到vps上(最后一次使用FileZilla了<3), 把指向这个php文件的url填写到WebHook页面的deploy_url里面, 并且点击测试, 之后回到vps上查看MySQL是否已经有log, 再在开发机试试commit&push, 你会发现, 代码已经悄然部署到vps上了.

腾讯云对象存储服务(cos)之PHP实践

云对象存储服务, BAT都有其业务, 本文选择腾讯是因为腾讯有每月免费流量, 而阿里和百度都是需要先付费后使用, 另外新兴的像七牛云存储, 也是很不错的解决方案.
贴代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
<?php
/**
* Simple Tencent COS SDK
* 2016/05/09
* Author: hldh214 <hldh214@gmail.com>
*/

// 配置项 start
$appid = '';
$bucket_name = '';
$dir_name = '';
$secretID = '';
$secretKey = '';
// 配置项 end

// 需要存储的资源url, 这里用百度logo来做演示
$pic_url = 'http://www.baidu.com/img/logo.gif';
// 获取文件名
$filename = end(explode('/', $pic_url));
// 构造上传url
$upload_url = "web.file.myqcloud.com/files/v1/$appid/$bucket_name/$dir_name/$filename";
// 设置过期时间
$exp = time() + 3600;
// 构造鉴权key
$sign = "a=$appid&b=$bucket_name&k=$secretID&e=$exp&t=" . time() . '&r=' . rand() . "&f=/$appid/$bucket_name/$dir_name/$filename";
$sign = base64_encode(hash_hmac('SHA1', $sign, $secretKey, true) . $sign);
// 构造post数据
$post_data = [
'op' => 'upload',
'filecontent' => file_get_contents($pic_url), // baidu logo
];
// 设置post的headers, 加入鉴权key
$header = [
'Content-Type: multipart/form-data',
'Authorization: ' . $sign,
];
// post
$ch = curl_init($upload_url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
$res = curl_exec($ch);
curl_close($ch);
$res = json_decode($res, true);
if (isset($res['data']['access_url'])) {
// 成功, 输出文件url
echo $res['data']['access_url'];
} else {
// 失败
echo $res;
}

PyQt5-Beginner-tutorial-part 3

原文链接

引言

这篇文章是前作的续篇(part 3), 为了达到最好的教学效果, 请务必先阅读前作.
在这篇续作当中, 我们将要开发一个简易文本阅读器, 简单起见, 这个阅读器只能阅读文本文档的内容, 但是, 只需要经过一些简单的修改就能成为一个全能的编辑器啦.

代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
import sys 
import os
from PyQt5.QtCore import *
from PyQt5.QtWidgets import *

class Notepad(QMainWindow):
def __init__(self):
super(Notepad, self).__init__()
self.initUI()

def initUI(self):
openAction = QAction('Open', self)
openAction.setShortcut('Ctrl+O')
openAction.setStatusTip('Open a file')
openAction.triggered.connect(self.openFile)

closeAction = QAction('Close', self)
closeAction.setShortcut('Ctrl+Q')
closeAction.setStatusTip('Close Notepad')
closeAction.triggered.connect(self.close)

menubar = self.menuBar()
fileMenu = menubar.addMenu('&File')
fileMenu.addAction(openAction)
fileMenu.addAction(closeAction)

self.textEdit = QTextEdit(self)
self.textEdit.setFocus()
self.textEdit.setReadOnly(True)

self.resize(700, 800)
self.setWindowTitle('Notepad')
self.setCentralWidget(self.textEdit)
self.show()

def openFile(self):
filename, _ = QFileDialog.getOpenFileName(self, 'Open File', os.getenv('HOME'))

fh = ''

if QFile.exists(filename):
fh = QFile(filename)

if not fh.open(QFile.ReadOnly):
QtGui.qApp.quit()

data = fh.readAll()
codec = QTextCodec.codecForUtfText(data)
unistr = codec.toUnicode(data)

tmp = ('Notepad: %s' % filename)
self.setWindowTitle(tmp)

self.textEdit.setText(unistr)

def main():
app = QApplication(sys.argv)
notepad = Notepad()
sys.exit(app.exec_())

if __name__ == '__main__':
main()

代码分析

其实你应该能看懂绝大部分的代码, 我们定义的类继承与 QMainWindow 这个基类, 这样可以让我们把菜单栏放置到窗体的顶部.
12-15行: 使用信号和槽来让对象互相之间进行通讯, 这是Qt众多核心特征之一, 当一个事件发生的时候, 将会发出一个信号, 而槽则是Python里面的可调用的对象, 如果一个信号和一个槽相连接的话, 这个槽所指的对象将会在收到信号的时候被调用, 反之亦然.
当按下 Open 的时候, 相当于给 openFile 这个对象发出了调用信号, 同样的, 通过键盘快捷键来完成这一事件同样在代码中定义了.
22-25行: 定义了一个菜单组件, 并且给这个菜单组件添加了两个动作, 这样一来, 用户将会看到一个 File 菜单, 快捷键是 Alt+F, 当其被点击的时候, 将会出现含有两个选项的菜单.
第37行: 弹出一个浏览文件的对话框, 其中第二个参数是浏览文件对话框的标题内容, 第三个参数则是默认打开的目录, 这个目录默认是这个代码所在目录.
41-42行: 检测这个文件是否存在, 如果存在则把其赋值给fh这个变量.
44-45行: 尝试打开这个文件, 如果打开失败则退出程序.
47-49行: 把这个文件对象里面的内容全部读取出来, 然后尝试设置文件编码, 我们最先尝试Unicode编码, 如果失败了则使用Latin-1编码.
51-52行: 把窗体的标题改为文件路径, 并在前面加上 ‘Notepad’ .

总结

我在底部留了一个状态栏的坑, 欢迎大家前来填坑~
希望你能喜欢这篇文章 :-)