9.3 Python处理HTML转义字符

[vieyahn2017]

https://www.cnblogs.com/xuxn/archive/2011/08/12/parse-html-escape-characters-in-python.html

[vieyahn2017]

抓网页数据经常遇到例如>或者 这种HTML转义符，抓到字符串里很是烦人。

比方说一个从网页中抓到的字符串

html = '<abc>'

用Python可以这样处理：

import HTMLParser
html_parser = HTMLParser.HTMLParser()
txt = html_parser.unescape(html) #这样就得到了txt = '<abc>'

如果还想转回去，可以这样：

import cgi
html = cgi.escape(txt) # 这样又回到了 html = '<abc&gt'

来回转的功能还分了两个模块实现，挺奇怪。没找到更优美的方法，欢迎补充哈~

[vieyahn2017]

"""
html字符转义 https://blog.csdn.net/qq_24122593/article/details/53485502

No. 文字表記    10進表記   16進表記   文字      Comment
001 "  "   "  " " "     quotation mark = APL quote
002 &   &   &  "&"     ampersand
003 &lt;    &#60;   &#x3C;  "<"     less-than sign
004 &gt;    &#62;   &#x3E;  ">"     greater-than sign
005 &nbsp;  &#160;  &#xA0;  " "     no-break space = non-breaking space
"""

[vieyahn2017]

我的代码

import HTMLParser
import re
import sys
import time
import datetime

reload(sys)
sys.setdefaultencoding('utf8')

html_parser = HTMLParser.HTMLParser()

file = open("console.txt", mode='r')
html = file.read()
txt = html_parser.unescape(html) 

# 替换console产生的VM268:14 这种字符串
txt = re.sub(r'VM(\d)+:(\d)+ ', '', txt)
print(txt)

#strtime = (datetime.datetime.now() - datetime.timedelta(hours=1)).strftime("%Y%m%d%H%M%S")
strtime = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
file2 = open("{}-exam.txt".format(strtime), mode='w')
file2.write(txt)

[vieyahn2017]

html字符转义 https://blog.csdn.net/qq_24122593/article/details/53485502

[vieyahn2017]

parse_console.py

#!/usr/bin/python
# -*- coding: utf-8 -*-

"""
html字符转义，直接使用HTMLParser.HTMLParser()
参看https://blog.csdn.net/qq_24122593/article/details/53485502

No. 文字表記    10進表記   16進表記   文字      Comment
001 "  "   "  " " "     quotation mark = APL quote
002 &   &   &  "&"     ampersand
003 &lt;    &#60;   &#x3C;  "<"     less-than sign
004 &gt;    &#62;   &#x3E;  ">"     greater-than sign
005 &nbsp;  &#160;  &#xA0;  " "     no-break space = non-breaking space
"""

import HTMLParser
import re
import sys
import time
import datetime

reload(sys)
sys.setdefaultencoding('utf8')

html_parser = HTMLParser.HTMLParser()

file = open("console.txt", mode='r')
txt = file.read()
# 替换console产生的VM268:14 这种字符串
txt = re.sub(r'VM(\d)+:(\d)+ ', '', txt)
print(txt)

html = html_parser.unescape(txt) 

#strtime = (datetime.datetime.now() - datetime.timedelta(hours=1)).strftime("%Y%m%d%H%M%S")
strtime = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
file2 = open("{}-exam.txt".format(strtime), mode='w')
file2.write(html)