Open billstore opened 9 years ago
This is a trick way in this project now. you need upload too files one is named "start.txt" and the other is "xxxx.py". in start.txt, the content should like this: python Tester.py
We suppose you named the xxxx.py to Tester.py like: from urllib import urlopen import time import re import random
url='http://127.0.0.1:8000/show/' regex='\d{4}[.:-]\d{1,2}[.:-]\d{1,2}'
def handle_context(context): pattern=re.compile(regex) with open('./context.txt','w') as file: for line in context: file.write('context>>>>'+line+'\n') m=pattern.search(line) if not m:continue file.write('group(0)>>>>'+m.group(0)+'\n')
def main(): file = open('./context.txt', 'w') for i in range(1000): value = random.random() value = 1000000.0*value/8.0 + random.random() file.write(str(value) + '\n') file.flush() time.sleep(1) file.close()
if name=='main': main()
you can upload the too files and have trial.
按照提示的内容编写了py文件进行了测试,还是发生了错误,请看看原因.
================以下为日志内容============================== root@ocpcm-srv:/opt/ptap/bin# python startAll.py ++++++++++++++++++++START SERVER+++++++++++++++++++++++ starting clusterService/MachineManager.py ... ++++++++++++++++++++START FRONTEND +++++++++++++++++++++++ starting webServer/server... starting clusterService/FrontendGuardServer ... start server ip = 10.4.54.36 port = 52813 REGISTER MACHINES LISTENER server finished Validating models...
0 errors found June 08, 2015 - 01:54:01 Django version 1.6.11, using settings 'webServer.settings' Starting development server at http://0.0.0.0:8000/ Quit the server with CONTROL-C. [08/Jun/2015 01:54:01] "GET /show/showtable?type=new&item=tests HTTP/1.1" 301 0 [08/Jun/2015 01:54:01] "GET /show/showtable/?type=new&item=tests HTTP/1.1" 200 10599 root@ocpcm-srv:/opt/ptap/bin# ++++++++++++++++++++++++++++++START BACKGROUND CLIENT ++++++++++++++++++++ starting clusterService/AgentClient.py ... start check start serve
[08/Jun/2015 01:54:06] "GET /show/run/?id=3 HTTP/1.1" 302 0 FrontendAgentSender run task : (1, [u'guojw/app/source/2.0/steven3/2015540801061433728446/', u'platform', 30L, 1, 10L, 4L, u'10.4.54.36', u'guojw/app/source/2.0/', -1, 14L, u'5']) RUN TEST FILESENDER CONNECT TO SERVER ( 10.4.54.36 , 62429 ) WRITEDONE FILESENDER SEND FILES FINISHED DeploymentControler run> popen args: from urllib import urlopen /opt/ptap//AllSource/ClientSourceCode/guojw/app/source/2.0/ from: can't read /var/mail/urllib START SOURCE CODE PROCESS PID : 22144 JOB DONE [08/Jun/2015 01:54:06] "GET /show/new/results HTTP/1.1" 200 12848
usage: perf record [
-e, --event <event> event selector. use 'perf list' to list available events
--filter <filter>
event filter
-p, --pid <n> record events on existing process id
-t, --tid <n> record events on existing thread id
-r, --realtime <n> collect data with this RT SCHED_FIFO priority
-D, --no-delay collect data without buffering
-R, --raw-samples collect raw sample records from all opened counters
-a, --all-cpus system-wide collection from all CPUs
-A, --append append to the output file to do incremental profiling
-C, --cpu <cpu> list of cpus to monitor
-f, --force overwrite existing data file (deprecated)
-c, --count <n> event period to sample
-o, --output <file> output file name
-i, --no-inherit child tasks do not inherit counters
-F, --freq <n> profile at this frequency
-m, --mmap-pages <n> number of mmap data pages
--group put the counters into a counter group
-g, --call-graph do call-graph (stack chain/backtrace) recording
-v, --verbose be more verbose (show counter open errors, etc)
-q, --quiet don't print any message
-s, --stat per thread counts
-d, --data Sample addresses
-T, --timestamp Sample timestamps
-P, --period Sample period
-n, --no-samples don't sample
-N, --no-buildid-cache
do not update the buildid cache
-B, --no-buildid do not collect buildids in perf.data
-G, --cgroup <name> monitor event in cgroup name only
Problems finding threads of monitor
usage: perf stat [
-e, --event <event> event selector. use 'perf list' to list available events
--filter <filter>
event filter
-i, --no-inherit child tasks do not inherit counters
-p, --pid <n> stat events on existing process id
-t, --tid <n> stat events on existing thread id
-a, --all-cpus system-wide collection from all CPUs
-g, --group put the counters into a counter group
-c, --scale scale/normalize counters
-v, --verbose be more verbose (show counter open errors, etc)
-r, --repeat <n> repeat command and print average + stddev (max: 100)
-n, --null null run - dont start any counters
-d, --detailed detailed run - start a lot of events
-S, --sync call sync() before starting a run
-B, --big-num print large numbers with thousands' separators
-C, --cpu <cpu> list of cpus to monitor in system-wide
-A, --no-aggr disable CPU count aggregation
-x, --field-separator <separator>
print counts with custom separator
-G, --cgroup <name> monitor event in cgroup name only
-o, --output <file> output file name
--append append to the output file
--log-fd <n> log output to fd, instead of stderr
failed to open AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015540801061433728446//Raw/Metrics/Hotspots/perf-20150608095416: No such file or directory chmod: cannot access ‘AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015540801061433728446//Raw/Metrics/Hotspots/perf-20150608095416’: No such file or directory
补充说明: 根据提示信息,已在服务器上安装了, sysstat, urllib等pkg.
@you-n-g help billstore to verify the problem and fixed it.
@billstore do you run the ptap with the "ROOT" user?
是以root运行ptap
@billstore 你看看这里 https://github.com/cloudswenable/ptap/tree/experiment#about-the-format-of-your-application-for-testing start.txt应该是包含了启动程序的shell命令,比如 python Hello.py 。 而不是直接将你写得 python脚本写到start.txt中(我在日志中看到 DeploymentControler run> popen args: from urllib import urlopen ,说明你把python脚本写到 start.txt了)
@you-n-g 上次测试确实是把python写到了start.txt. 现在已经改过来了,但是执行还是提示如下错误:
=================Log========================= root@ocpcm-srv:/opt/ptap/AllSource/SourceCode/guojw/app/source/2.0# python /opt/ptap/bin/startAll.py ++++++++++++++++++++START SERVER+++++++++++++++++++++++ starting clusterService/MachineManager.py ... ++++++++++++++++++++START FRONTEND +++++++++++++++++++++++ starting webServer/server... starting clusterService/FrontendGuardServer ... start server ip = 10.4.54.36 port = 35289 REGISTER MACHINES LISTENER server finished Validating models...
0 errors found June 09, 2015 - 03:42:43 Django version 1.6.11, using settings 'webServer.settings' Starting development server at http://0.0.0.0:8000/ Quit the server with CONTROL-C. [09/Jun/2015 03:42:45] "GET /show/showtable?type=new&item=tests HTTP/1.1" 301 0 [09/Jun/2015 03:42:45] "GET /show/showtable/?type=new&item=tests HTTP/1.1" 200 10599 root@ocpcm-srv:/opt/ptap/AllSource/SourceCode/guojw/app/source/2.0# ++++++++++++++++++++++++++++++START BACKGROUND CLIENT ++++++++++++++++++++ starting clusterService/AgentClient.py ... start check start serve
[09/Jun/2015 03:42:51] "GET /show/run/?id=3 HTTP/1.1" 302 0 FrontendAgentSender run task : (1, [u'guojw/app/source/2.0/steven3/2015420903061433821371/', u'platform', 30L, 1, 10L, 4L, u'10.4.54.36', u'guojw/app/source/2.0/', -1, 20L, u'5']) RUN TEST FILESENDER CONNECT TO SERVER ( 10.4.54.36 , 61331 ) WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE [09/Jun/2015 03:42:51] "GET /show/new/results HTTP/1.1" 200 11698 WRITEDONE FILESENDER SEND FILES FINISHED DeploymentControler run> popen args: python app.py /opt/ptap//AllSource/ClientSourceCode/guojw/app/source/2.0/ START SOURCE CODE PROCESS PID : 31463 JOB DONE Problems finding threads of monitor
usage: perf stat [
-e, --event <event> event selector. use 'perf list' to list available events
--filter <filter>
event filter
-i, --no-inherit child tasks do not inherit counters
-p, --pid <n> stat events on existing process id
-t, --tid <n> stat events on existing thread id
-a, --all-cpus system-wide collection from all CPUs
-g, --group put the counters into a counter group
-c, --scale scale/normalize counters
-v, --verbose be more verbose (show counter open errors, etc)
-r, --repeat <n> repeat command and print average + stddev (max: 100)
-n, --null null run - dont start any counters
-d, --detailed detailed run - start a lot of events
-S, --sync call sync() before starting a run
-B, --big-num print large numbers with thousands' separators
-C, --cpu <cpu> list of cpus to monitor in system-wide
-A, --no-aggr disable CPU count aggregation
-x, --field-separator <separator>
print counts with custom separator
-G, --cgroup <name> monitor event in cgroup name only
-o, --output <file> output file name
--append append to the output file
--log-fd <n> log output to fd, instead of stderr
usage: perf record [
-e, --event <event> event selector. use 'perf list' to list available events
--filter <filter>
event filter
-p, --pid <n> record events on existing process id
-t, --tid <n> record events on existing thread id
-r, --realtime <n> collect data with this RT SCHED_FIFO priority
-D, --no-delay collect data without buffering
-R, --raw-samples collect raw sample records from all opened counters
-a, --all-cpus system-wide collection from all CPUs
-A, --append append to the output file to do incremental profiling
-C, --cpu <cpu> list of cpus to monitor
-f, --force overwrite existing data file (deprecated)
-c, --count <n> event period to sample
-o, --output <file> output file name
-i, --no-inherit child tasks do not inherit counters
-F, --freq <n> profile at this frequency
-m, --mmap-pages <n> number of mmap data pages
--group put the counters into a counter group
-g, --call-graph do call-graph (stack chain/backtrace) recording
-v, --verbose be more verbose (show counter open errors, etc)
-q, --quiet don't print any message
-s, --stat per thread counts
-d, --data Sample addresses
-T, --timestamp Sample timestamps
-P, --period Sample period
-n, --no-samples don't sample
-N, --no-buildid-cache
do not update the buildid cache
-B, --no-buildid do not collect buildids in perf.data
-G, --cgroup <name> monitor event in cgroup name only
failed to open AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015420903061433821371//Raw/Metrics/Hotspots/perf-20150609114302: No such file or directory chmod: cannot access ‘AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015420903061433821371//Raw/Metrics/Hotspots/perf-20150609114302’: No such file or directory Problems finding threads of monitor
@billstore 因为性能监控是一个过程, 所以你的脚本如果只是时间太短, 那么perf是来不及提取运行时的数据的。 这次报错是:Problems finding threads of monitor 也就是 是 perf去抓数据时, 进程已经执行完毕了。 所以你的程序执行时间得长一点
执行“stop”命令时,输出以下信息,点击“show”页面图标横轴有显示“task-clock(CPU Utilized), CPI”等内ron,但是二维空间内没有数据显示。
failed to open AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015480903061433821684//Raw/Metrics/Hotspots/perf-20150609114824: No such file or directory chmod: cannot access ‘AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015480903061433821684//Raw/Metrics/Hotspots/perf-20150609114824’: No such file or directory Exception in thread Thread-45: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/opt/ptap/clientService/Processor.py", line 34, in run results = self.handle(self.job or self.config.job_info) File "/opt/ptap/clientService/HotspotsProcessor.py", line 141, in handle for file in os.listdir(inputPath): OSError: [Errno 2] No such file or directory: '/opt/ptap/AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015480903061433821684//Raw/Metrics/Hotspots/'
Exception in thread Thread-42: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/opt/ptap/clientService/Processor.py", line 34, in run results = self.handle(self.job or self.config.job_info) File "/opt/ptap/clientService/SARProcessor.py", line 198, in handle for file in os.listdir(inputPath): OSError: [Errno 2] No such file or directory: '/opt/ptap/AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015480903061433821684//Raw/Metrics/SAR/'
FILESENDER CONNECT TO SERVER ( 10.4.54.36 , 32489 ) WRITEDONE WRITEDONE WRITEDONE FILESENDER SEND FILES FINISHED [09/Jun/2015 03:49:20] "GET /show/showresult/?type=results&id=21 HTTP/1.1" 200 70 QUERY A RESULT OVERVIEW
错误信息“usage: perf stat [] [] ...."是在点完“run”按钮后就输出了。测试持续大约6分钟。点stop时输出了上面的内容
@billstore
现在还有没有 Problems finding threads of monitor
这个报错出来?
如果有的话, 应该还是进程没有起来。 这个错误是由于根据pid 找不到 要监控的进程导致的。
比如你看到如下报错
START SOURCE CODE PROCESS PID : 31463
JOB DONE
Problems finding threads of monitor
usage: perf stat [] []
那么可以试着先检查一下 PID为31463 的进程是否存在, 如果不存在的话,可能是这时程序没起来。
我在new页面自定义了一个test, 上传了一个文件abc.txt到测试机器, 但是运行这个测试时,在测试机器提示'start.txt'找不到,经过几个测试发现,不上传什么文件都提示start.txt找不到。