cloudswenable / ptap

ptap(performance tracking and analysis platform) is a platform help you to automate test your program, collect some performance data and give out test report
Other
4 stars 6 forks source link

Error messge after running a test on the page 'new' #12

Open billstore opened 9 years ago

billstore commented 9 years ago

我在new页面自定义了一个test, 上传了一个文件abc.txt到测试机器, 但是运行这个测试时,在测试机器提示'start.txt'找不到,经过几个测试发现,不上传什么文件都提示start.txt找不到。

cloudswenable commented 9 years ago

This is a trick way in this project now. you need upload too files one is named "start.txt" and the other is "xxxx.py". in start.txt, the content should like this: python Tester.py

We suppose you named the xxxx.py to Tester.py like: from urllib import urlopen import time import re import random

url='http://127.0.0.1:8000/show/' regex='\d{4}[.:-]\d{1,2}[.:-]\d{1,2}'

def handle_context(context): pattern=re.compile(regex) with open('./context.txt','w') as file: for line in context: file.write('context>>>>'+line+'\n') m=pattern.search(line) if not m:continue file.write('group(0)>>>>'+m.group(0)+'\n')

def main(): file = open('./context.txt', 'w') for i in range(1000): value = random.random() value = 1000000.0*value/8.0 + random.random() file.write(str(value) + '\n') file.flush() time.sleep(1) file.close()

if name=='main': main()

you can upload the too files and have trial.

billstore commented 9 years ago

按照提示的内容编写了py文件进行了测试,还是发生了错误,请看看原因.

================以下为日志内容============================== root@ocpcm-srv:/opt/ptap/bin# python startAll.py ++++++++++++++++++++START SERVER+++++++++++++++++++++++ starting clusterService/MachineManager.py ... ++++++++++++++++++++START FRONTEND +++++++++++++++++++++++ starting webServer/server... starting clusterService/FrontendGuardServer ... start server ip = 10.4.54.36 port = 52813 REGISTER MACHINES LISTENER server finished Validating models...

0 errors found June 08, 2015 - 01:54:01 Django version 1.6.11, using settings 'webServer.settings' Starting development server at http://0.0.0.0:8000/ Quit the server with CONTROL-C. [08/Jun/2015 01:54:01] "GET /show/showtable?type=new&item=tests HTTP/1.1" 301 0 [08/Jun/2015 01:54:01] "GET /show/showtable/?type=new&item=tests HTTP/1.1" 200 10599 root@ocpcm-srv:/opt/ptap/bin# ++++++++++++++++++++++++++++++START BACKGROUND CLIENT ++++++++++++++++++++ starting clusterService/AgentClient.py ... start check start serve

[08/Jun/2015 01:54:06] "GET /show/run/?id=3 HTTP/1.1" 302 0 FrontendAgentSender run task : (1, [u'guojw/app/source/2.0/steven3/2015540801061433728446/', u'platform', 30L, 1, 10L, 4L, u'10.4.54.36', u'guojw/app/source/2.0/', -1, 14L, u'5']) RUN TEST FILESENDER CONNECT TO SERVER ( 10.4.54.36 , 62429 ) WRITEDONE FILESENDER SEND FILES FINISHED DeploymentControler run> popen args: from urllib import urlopen /opt/ptap//AllSource/ClientSourceCode/guojw/app/source/2.0/ from: can't read /var/mail/urllib START SOURCE CODE PROCESS PID : 22144 JOB DONE [08/Jun/2015 01:54:06] "GET /show/new/results HTTP/1.1" 200 12848

usage: perf record [] [] or: perf record [] -- []

-e, --event <event>   event selector. use 'perf list' to list available events
    --filter <filter>
                      event filter
-p, --pid <n>         record events on existing process id
-t, --tid <n>         record events on existing thread id
-r, --realtime <n>    collect data with this RT SCHED_FIFO priority
-D, --no-delay        collect data without buffering
-R, --raw-samples     collect raw sample records from all opened counters
-a, --all-cpus        system-wide collection from all CPUs
-A, --append          append to the output file to do incremental profiling
-C, --cpu <cpu>       list of cpus to monitor
-f, --force           overwrite existing data file (deprecated)
-c, --count <n>       event period to sample
-o, --output <file>   output file name
-i, --no-inherit      child tasks do not inherit counters
-F, --freq <n>        profile at this frequency
-m, --mmap-pages <n>  number of mmap data pages
    --group           put the counters into a counter group
-g, --call-graph      do call-graph (stack chain/backtrace) recording
-v, --verbose         be more verbose (show counter open errors, etc)
-q, --quiet           don't print any message
-s, --stat            per thread counts
-d, --data            Sample addresses
-T, --timestamp       Sample timestamps
-P, --period          Sample period
-n, --no-samples      don't sample
-N, --no-buildid-cache
                      do not update the buildid cache
-B, --no-buildid      do not collect buildids in perf.data
-G, --cgroup <name>   monitor event in cgroup name only

Problems finding threads of monitor

usage: perf stat [] []

-e, --event <event>   event selector. use 'perf list' to list available events
    --filter <filter>
                      event filter
-i, --no-inherit      child tasks do not inherit counters
-p, --pid <n>         stat events on existing process id
-t, --tid <n>         stat events on existing thread id
-a, --all-cpus        system-wide collection from all CPUs
-g, --group           put the counters into a counter group
-c, --scale           scale/normalize counters
-v, --verbose         be more verbose (show counter open errors, etc)
-r, --repeat <n>      repeat command and print average + stddev (max: 100)
-n, --null            null run - dont start any counters
-d, --detailed        detailed run - start a lot of events
-S, --sync            call sync() before starting a run
-B, --big-num         print large numbers with thousands' separators
-C, --cpu <cpu>       list of cpus to monitor in system-wide
-A, --no-aggr         disable CPU count aggregation
-x, --field-separator <separator>
                      print counts with custom separator
-G, --cgroup <name>   monitor event in cgroup name only
-o, --output <file>   output file name
    --append          append to the output file
    --log-fd <n>      log output to fd, instead of stderr

failed to open AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015540801061433728446//Raw/Metrics/Hotspots/perf-20150608095416: No such file or directory chmod: cannot access ‘AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015540801061433728446//Raw/Metrics/Hotspots/perf-20150608095416’: No such file or directory

billstore commented 9 years ago

补充说明: 根据提示信息,已在服务器上安装了, sysstat, urllib等pkg.

cloudswenable commented 9 years ago

@you-n-g help billstore to verify the problem and fixed it.

cloudswenable commented 9 years ago

@billstore do you run the ptap with the "ROOT" user?

billstore commented 9 years ago

是以root运行ptap

you-n-g commented 9 years ago

@billstore 你看看这里 https://github.com/cloudswenable/ptap/tree/experiment#about-the-format-of-your-application-for-testing start.txt应该是包含了启动程序的shell命令,比如 python Hello.py 。 而不是直接将你写得 python脚本写到start.txt中(我在日志中看到 DeploymentControler run> popen args: from urllib import urlopen ,说明你把python脚本写到 start.txt了)

billstore commented 9 years ago

@you-n-g 上次测试确实是把python写到了start.txt. 现在已经改过来了,但是执行还是提示如下错误:

=================Log========================= root@ocpcm-srv:/opt/ptap/AllSource/SourceCode/guojw/app/source/2.0# python /opt/ptap/bin/startAll.py ++++++++++++++++++++START SERVER+++++++++++++++++++++++ starting clusterService/MachineManager.py ... ++++++++++++++++++++START FRONTEND +++++++++++++++++++++++ starting webServer/server... starting clusterService/FrontendGuardServer ... start server ip = 10.4.54.36 port = 35289 REGISTER MACHINES LISTENER server finished Validating models...

0 errors found June 09, 2015 - 03:42:43 Django version 1.6.11, using settings 'webServer.settings' Starting development server at http://0.0.0.0:8000/ Quit the server with CONTROL-C. [09/Jun/2015 03:42:45] "GET /show/showtable?type=new&item=tests HTTP/1.1" 301 0 [09/Jun/2015 03:42:45] "GET /show/showtable/?type=new&item=tests HTTP/1.1" 200 10599 root@ocpcm-srv:/opt/ptap/AllSource/SourceCode/guojw/app/source/2.0# ++++++++++++++++++++++++++++++START BACKGROUND CLIENT ++++++++++++++++++++ starting clusterService/AgentClient.py ... start check start serve

[09/Jun/2015 03:42:51] "GET /show/run/?id=3 HTTP/1.1" 302 0 FrontendAgentSender run task : (1, [u'guojw/app/source/2.0/steven3/2015420903061433821371/', u'platform', 30L, 1, 10L, 4L, u'10.4.54.36', u'guojw/app/source/2.0/', -1, 20L, u'5']) RUN TEST FILESENDER CONNECT TO SERVER ( 10.4.54.36 , 61331 ) WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE WRITEDONE [09/Jun/2015 03:42:51] "GET /show/new/results HTTP/1.1" 200 11698 WRITEDONE FILESENDER SEND FILES FINISHED DeploymentControler run> popen args: python app.py /opt/ptap//AllSource/ClientSourceCode/guojw/app/source/2.0/ START SOURCE CODE PROCESS PID : 31463 JOB DONE Problems finding threads of monitor

usage: perf stat [] []

-e, --event <event>   event selector. use 'perf list' to list available events
    --filter <filter>
                      event filter
-i, --no-inherit      child tasks do not inherit counters
-p, --pid <n>         stat events on existing process id
-t, --tid <n>         stat events on existing thread id
-a, --all-cpus        system-wide collection from all CPUs
-g, --group           put the counters into a counter group
-c, --scale           scale/normalize counters
-v, --verbose         be more verbose (show counter open errors, etc)
-r, --repeat <n>      repeat command and print average + stddev (max: 100)
-n, --null            null run - dont start any counters
-d, --detailed        detailed run - start a lot of events
-S, --sync            call sync() before starting a run
-B, --big-num         print large numbers with thousands' separators
-C, --cpu <cpu>       list of cpus to monitor in system-wide
-A, --no-aggr         disable CPU count aggregation
-x, --field-separator <separator>
                      print counts with custom separator
-G, --cgroup <name>   monitor event in cgroup name only
-o, --output <file>   output file name
    --append          append to the output file
    --log-fd <n>      log output to fd, instead of stderr

usage: perf record [] [] or: perf record [] -- []

-e, --event <event>   event selector. use 'perf list' to list available events
    --filter <filter>
                      event filter
-p, --pid <n>         record events on existing process id
-t, --tid <n>         record events on existing thread id
-r, --realtime <n>    collect data with this RT SCHED_FIFO priority
-D, --no-delay        collect data without buffering
-R, --raw-samples     collect raw sample records from all opened counters
-a, --all-cpus        system-wide collection from all CPUs
-A, --append          append to the output file to do incremental profiling
-C, --cpu <cpu>       list of cpus to monitor
-f, --force           overwrite existing data file (deprecated)
-c, --count <n>       event period to sample
-o, --output <file>   output file name
-i, --no-inherit      child tasks do not inherit counters
-F, --freq <n>        profile at this frequency
-m, --mmap-pages <n>  number of mmap data pages
    --group           put the counters into a counter group
-g, --call-graph      do call-graph (stack chain/backtrace) recording
-v, --verbose         be more verbose (show counter open errors, etc)
-q, --quiet           don't print any message
-s, --stat            per thread counts
-d, --data            Sample addresses
-T, --timestamp       Sample timestamps
-P, --period          Sample period
-n, --no-samples      don't sample
-N, --no-buildid-cache
                      do not update the buildid cache
-B, --no-buildid      do not collect buildids in perf.data
-G, --cgroup <name>   monitor event in cgroup name only

failed to open AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015420903061433821371//Raw/Metrics/Hotspots/perf-20150609114302: No such file or directory chmod: cannot access ‘AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015420903061433821371//Raw/Metrics/Hotspots/perf-20150609114302’: No such file or directory Problems finding threads of monitor

you-n-g commented 9 years ago

@billstore 因为性能监控是一个过程, 所以你的脚本如果只是时间太短, 那么perf是来不及提取运行时的数据的。 这次报错是:Problems finding threads of monitor 也就是 是 perf去抓数据时, 进程已经执行完毕了。 所以你的程序执行时间得长一点

billstore commented 9 years ago

执行“stop”命令时,输出以下信息,点击“show”页面图标横轴有显示“task-clock(CPU Utilized), CPI”等内ron,但是二维空间内没有数据显示。

failed to open AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015480903061433821684//Raw/Metrics/Hotspots/perf-20150609114824: No such file or directory chmod: cannot access ‘AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015480903061433821684//Raw/Metrics/Hotspots/perf-20150609114824’: No such file or directory Exception in thread Thread-45: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/opt/ptap/clientService/Processor.py", line 34, in run results = self.handle(self.job or self.config.job_info) File "/opt/ptap/clientService/HotspotsProcessor.py", line 141, in handle for file in os.listdir(inputPath): OSError: [Errno 2] No such file or directory: '/opt/ptap/AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015480903061433821684//Raw/Metrics/Hotspots/'

Exception in thread Thread-42: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/opt/ptap/clientService/Processor.py", line 34, in run results = self.handle(self.job or self.config.job_info) File "/opt/ptap/clientService/SARProcessor.py", line 198, in handle for file in os.listdir(inputPath): OSError: [Errno 2] No such file or directory: '/opt/ptap/AllSource/ClientOutput/guojw/app/source/2.0/steven3/2015480903061433821684//Raw/Metrics/SAR/'

FILESENDER CONNECT TO SERVER ( 10.4.54.36 , 32489 ) WRITEDONE WRITEDONE WRITEDONE FILESENDER SEND FILES FINISHED [09/Jun/2015 03:49:20] "GET /show/showresult/?type=results&id=21 HTTP/1.1" 200 70 QUERY A RESULT OVERVIEW

billstore commented 9 years ago

错误信息“usage: perf stat [] [] ...."是在点完“run”按钮后就输出了。测试持续大约6分钟。点stop时输出了上面的内容

you-n-g commented 9 years ago

@billstore

现在还有没有 Problems finding threads of monitor 这个报错出来? 如果有的话, 应该还是进程没有起来。 这个错误是由于根据pid 找不到 要监控的进程导致的。

比如你看到如下报错

START SOURCE CODE PROCESS PID : 31463
JOB DONE
Problems finding threads of monitor

usage: perf stat [] []

那么可以试着先检查一下 PID为31463 的进程是否存在, 如果不存在的话,可能是这时程序没起来。