Closed GoogleCodeExporter closed 9 years ago
Could you provide more detail?
Original comment by joe.gregorio@gmail.com
on 3 Dec 2010 at 12:46
I create 4 threads, each thread listens to a queue and receives from her ga:
tableId. Once the flow gets his ga: tableId, he gets all the pages to the site
the previous day, then all the keywords for those pages, ie gets something like
an object {'page_path': {'keyword1': {'bounces': 0, 'exits': 0 ...}, ...}, ...}
The problem is that some keywords for a particular path (most often it is the
path "/") are in another thread. This means that the Key words which must be
received in the first thread turn into an object that belongs to the second
thread.
In the end everything is mixed, BUT the stream are isolated from each other.
My manual traceback:
Thread2 get data from queue: ['ga:13538458', 'moda.nesterova.ru', False]
Thread4 get data from queue: ['ga:33075170', 'stal-kovka.ru', False]
Thread1 get data from queue: ['ga:24424890', 'belorusof.ru', False]
Thread3 get data from queue: ['ga:32585455', 'undorstroy.ru', False]
Thread4: sent request = {'metrics': 'ga:pageviews', 'sort': '-ga:pageviews',
'max-results': '5000', 'dimensions': 'ga:pagePath', 'filters': 'ga:visits>0',
'start-date': '2010-12-02', 'start-index': '1', 'end-date': '2010-12-02',
'ids': 'ga:32585455'}
Thread4: response (feed) = <ns0:feed ns1:etag="W/"DkQNSHcycSp7I2A9Wx9SE04.""
ns1:kind="analytics#data" xmlns:ns0="http://www.w3.org/2005/Atom"
xmlns:ns1="http://schemas.google.com/g/2005"><ns0:id>http://www.google.com/analy
tics/feeds/data?ids=ga:33075170&dimensions=ga:pagePath&metrics=ga:pageviews&filt
ers=ga:visits%3E0&start-date=2010-12-02&end-date=2010-12-02</ns0:id><ns2:dataSou
rce
xmlns:ns2="http://schemas.google.com/analytics/2009"><ns2:tableName>stal-kovka.r
u</ns2:tableName><ns2:tableId>ga:33075170</ns2:tableId><ns2:property
name="ga:profileId" value="33075170" /><ns2:property name="ga:webPropertyId"
value="UA-9496597-38" /><ns2:property name="ga:accountName" value="s-t-u"
/></ns2:dataSource><ns2:aggregates
xmlns:ns2="http://schemas.google.com/analytics/2009"><ns2:metric
confidenceInterval="0.0" name="ga:pageviews" type="integer" value="0"
/></ns2:aggregates><ns0:updated>2010-12-02T15:59:59.999-08:00</ns0:updated><ns2:
containsSampledData
xmlns:ns2="http://schemas.google.com/analytics/2009">false</ns2:containsSampledD
ata><ns0:generator version="1.0">Google
Analytics</ns0:generator><ns2:startIndex
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">1</ns2:startIndex><ns0:title
>Google Analytics Data for Profile
33075170</ns0:title><ns0:author><ns0:name>Google
Analytics</ns0:name></ns0:author><ns0:link
href="https://www.google.com/analytics/feeds/data?start-index=1&max-results=5000
&sort=-ga%3Apageviews&end-date=2010-12-02&start-date=2010-12-02&metrics=ga%3Apag
eviews&ids=ga%3A33075170&dimensions=ga%3ApagePath&filters=ga%3Avisits%3E0"
rel="self" type="application/atom+xml" /><ns2:endDate
xmlns:ns2="http://schemas.google.com/analytics/2009">2010-12-02</ns2:endDate><ns
2:totalResults
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">0</ns2:totalResults><ns2:sta
rtDate
xmlns:ns2="http://schemas.google.com/analytics/2009">2010-12-02</ns2:startDate><
ns2:itemsPerPage
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">5000</ns2:itemsPerPage></ns0
:feed>
Thread4: result object = {}
Thread4 get data from queue: ['ga:17030176', 'visionvip.biz', False]
Thread1: sent request = {'metrics': 'ga:pageviews', 'sort': '-ga:pageviews',
'max-results': '5000', 'dimensions': 'ga:pagePath', 'filters': 'ga:visits>0',
'start-date': '2010-12-02', 'start-index': '1', 'end-date': '2010-12-02',
'ids': 'ga:17030176'}
Thread1: response (feed) = <ns0:feed ns1:etag="W/"DkQNSHcycSp7I2A9Wx9SE04.""
ns1:kind="analytics#data" xmlns:ns0="http://www.w3.org/2005/Atom"
xmlns:ns1="http://schemas.google.com/g/2005"><ns0:id>http://www.google.com/analy
tics/feeds/data?ids=ga:24424890&dimensions=ga:pagePath&metrics=ga:pageviews&filt
ers=ga:visits%3E0&start-date=2010-12-02&end-date=2010-12-02</ns0:id><ns2:dataSou
rce
xmlns:ns2="http://schemas.google.com/analytics/2009"><ns2:tableName>www.beloruso
f.ru</ns2:tableName><ns2:tableId>ga:24424890</ns2:tableId><ns2:property
name="ga:profileId" value="24424890" /><ns2:property name="ga:webPropertyId"
value="UA-9443824-27" /><ns2:property name="ga:accountName" value="a-b-c"
/></ns2:dataSource><ns2:aggregates
xmlns:ns2="http://schemas.google.com/analytics/2009"><ns2:metric
confidenceInterval="0.0" name="ga:pageviews" type="integer" value="48"
/></ns2:aggregates><ns0:updated>2010-12-02T15:59:59.999-08:00</ns0:updated><ns2:
containsSampledData
xmlns:ns2="http://schemas.google.com/analytics/2009">false</ns2:containsSampledD
ata><ns0:entry ns1:etag="W/"AkcEQX47eSp7I2A9Wx9SEkk.""
ns1:kind="analytics#datarow"><ns0:id>http://www.google.com/analytics/feeds/data?
ids=ga:24424890&ga:pagePath=/index.php?page%3Dtseny&filters=ga:visits%3E0&start-
date=2010-12-02&end-date=2010-12-02</ns0:id><ns0:updated>2010-12-01T16:00:00.001
-08:00</ns0:updated><ns2:dimension name="ga:pagePath"
value="/index.php?page=tseny"
xmlns:ns2="http://schemas.google.com/analytics/2009"
/><ns0:title>ga:pagePath=/index.php?page=tseny</ns0:title><ns2:metric
confidenceInterval="0.0" name="ga:pageviews" type="integer" value="16"
xmlns:ns2="http://schemas.google.com/analytics/2009" /><ns0:link
href="http://www.google.com/analytics" rel="alternate" type="text/html"
/></ns0:entry><ns0:entry ns1:etag="W/"AkcEQX47eSp7I2A9Wx9SEkk.""
ns1:kind="analytics#datarow"><ns0:id>http://www.google.com/analytics/feeds/data?
ids=ga:24424890&ga:pagePath=/index.php?page%3Dtehnologiya-ustrojstva-styazhki-po
lov&filters=ga:visits%3E0&start-date=2010-12-02&end-date=2010-12-02</ns0:id><ns0
:updated>2010-12-01T16:00:00.001-08:00</ns0:updated><ns2:dimension
name="ga:pagePath"
value="/index.php?page=tehnologiya-ustrojstva-styazhki-polov"
xmlns:ns2="http://schemas.google.com/analytics/2009"
/><ns0:title>ga:pagePath=/index.php?page=tehnologiya-ustrojstva-styazhki-polov</
ns0:title><ns2:metric confidenceInterval="0.0" name="ga:pageviews"
type="integer" value="15" xmlns:ns2="http://schemas.google.com/analytics/2009"
/><ns0:link href="http://www.google.com/analytics" rel="alternate"
type="text/html" /></ns0:entry><ns0:entry
ns1:etag="W/"AkcEQX47eSp7I2A9Wx9SEkk.""
ns1:kind="analytics#datarow"><ns0:id>http://www.google.com/analytics/feeds/data?
ids=ga:24424890&ga:pagePath=/&filters=ga:visits%3E0&start-date=2010-12-02&end-da
te=2010-12-02</ns0:id><ns0:updated>2010-12-01T16:00:00.001-08:00</ns0:updated><n
s2:dimension name="ga:pagePath" value="/"
xmlns:ns2="http://schemas.google.com/analytics/2009"
/><ns0:title>ga:pagePath=/</ns0:title><ns2:metric confidenceInterval="0.0"
name="ga:pageviews" type="integer" value="8"
xmlns:ns2="http://schemas.google.com/analytics/2009" /><ns0:link
href="http://www.google.com/analytics" rel="alternate" type="text/html"
/></ns0:entry><ns0:entry ns1:etag="W/"AkcEQX47eSp7I2A9Wx9SEkk.""
ns1:kind="analytics#datarow"><ns0:id>http://www.google.com/analytics/feeds/data?
ids=ga:24424890&ga:pagePath=/index.php?page%3Dkontakty&filters=ga:visits%3E0&sta
rt-date=2010-12-02&end-date=2010-12-02</ns0:id><ns0:updated>2010-12-01T16:00:00.
001-08:00</ns0:updated><ns2:dimension name="ga:pagePath"
value="/index.php?page=kontakty"
xmlns:ns2="http://schemas.google.com/analytics/2009"
/><ns0:title>ga:pagePath=/index.php?page=kontakty</ns0:title><ns2:metric
confidenceInterval="0.0" name="ga:pageviews" type="integer" value="4"
xmlns:ns2="http://schemas.google.com/analytics/2009" /><ns0:link
href="http://www.google.com/analytics" rel="alternate" type="text/html"
/></ns0:entry><ns0:entry ns1:etag="W/"AkcEQX47eSp7I2A9Wx9SEkk.""
ns1:kind="analytics#datarow"><ns0:id>http://www.google.com/analytics/feeds/data?
ids=ga:24424890&ga:pagePath=/index.php?page%3Dustrojstvo-styazhki-pola&filters=g
a:visits%3E0&start-date=2010-12-02&end-date=2010-12-02</ns0:id><ns0:updated>2010
-12-01T16:00:00.001-08:00</ns0:updated><ns2:dimension name="ga:pagePath"
value="/index.php?page=ustrojstvo-styazhki-pola"
xmlns:ns2="http://schemas.google.com/analytics/2009"
/><ns0:title>ga:pagePath=/index.php?page=ustrojstvo-styazhki-pola</ns0:title><ns
2:metric confidenceInterval="0.0" name="ga:pageviews" type="integer" value="3"
xmlns:ns2="http://schemas.google.com/analytics/2009" /><ns0:link
href="http://www.google.com/analytics" rel="alternate" type="text/html"
/></ns0:entry><ns0:entry ns1:etag="W/"AkcEQX47eSp7I2A9Wx9SEkk.""
ns1:kind="analytics#datarow"><ns0:id>http://www.google.com/analytics/feeds/data?
ids=ga:24424890&ga:pagePath=/index.php?page%3Dstyazhka-polov&filters=ga:visits%3
E0&start-date=2010-12-02&end-date=2010-12-02</ns0:id><ns0:updated>2010-12-01T16:
00:00.001-08:00</ns0:updated><ns2:dimension name="ga:pagePath"
value="/index.php?page=styazhka-polov"
xmlns:ns2="http://schemas.google.com/analytics/2009"
/><ns0:title>ga:pagePath=/index.php?page=styazhka-polov</ns0:title><ns2:metric
confidenceInterval="0.0" name="ga:pageviews" type="integer" value="2"
xmlns:ns2="http://schemas.google.com/analytics/2009" /><ns0:link
href="http://www.google.com/analytics" rel="alternate" type="text/html"
/></ns0:entry><ns0:generator version="1.0">Google
Analytics</ns0:generator><ns2:startIndex
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">1</ns2:startIndex><ns0:title
>Google Analytics Data for Profile
24424890</ns0:title><ns0:author><ns0:name>Google
Analytics</ns0:name></ns0:author><ns0:link
href="https://www.google.com/analytics/feeds/data?start-index=1&max-results=5000
&sort=-ga%3Apageviews&end-date=2010-12-02&start-date=2010-12-02&metrics=ga%3Apag
eviews&ids=ga%3A24424890&dimensions=ga%3ApagePath&filters=ga%3Avisits%3E0"
rel="self" type="application/atom+xml" /><ns2:endDate
xmlns:ns2="http://schemas.google.com/analytics/2009">2010-12-02</ns2:endDate><ns
2:totalResults
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">6</ns2:totalResults><ns2:sta
rtDate
xmlns:ns2="http://schemas.google.com/analytics/2009">2010-12-02</ns2:startDate><
ns2:itemsPerPage
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">5000</ns2:itemsPerPage></ns0
:feed>
Thread2: sent request = {'metrics': 'ga:pageviews', 'sort': '-ga:pageviews',
'max-results': '5000', 'dimensions': 'ga:pagePath', 'filters': 'ga:visits>0',
'start-date': '2010-12-02', 'start-index': '1', 'end-date': '2010-12-02',
'ids': 'ga:17030176'}
Thread2: response (feed) = <ns0:feed ns1:etag="W/"DkQNSHcycSp7I2A9Wx9SE04.""
ns1:kind="analytics#data" xmlns:ns0="http://www.w3.org/2005/Atom"
xmlns:ns1="http://schemas.google.com/g/2005"><ns0:id>http://www.google.com/analy
tics/feeds/data?ids=ga:13538458&dimensions=ga:pagePath&metrics=ga:pageviews&filt
ers=ga:visits%3E0&start-date=2010-12-02&end-date=2010-12-02</ns0:id><ns2:dataSou
rce
xmlns:ns2="http://schemas.google.com/analytics/2009"><ns2:tableName>moda.nestero
va.ru</ns2:tableName><ns2:tableId>ga:13538458</ns2:tableId><ns2:property
name="ga:profileId" value="13538458" /><ns2:property name="ga:webPropertyId"
value="UA-4618787-59" /><ns2:property name="ga:accountName" value="_ALTEGroup
2" /></ns2:dataSource><ns2:aggregates
xmlns:ns2="http://schemas.google.com/analytics/2009"><ns2:metric
confidenceInterval="0.0" name="ga:pageviews" type="integer" value="0"
/></ns2:aggregates><ns0:updated>2010-12-02T15:59:59.999-08:00</ns0:updated><ns2:
containsSampledData
xmlns:ns2="http://schemas.google.com/analytics/2009">false</ns2:containsSampledD
ata><ns0:generator version="1.0">Google
Analytics</ns0:generator><ns2:startIndex
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">1</ns2:startIndex><ns0:title
>Google Analytics Data for Profile
13538458</ns0:title><ns0:author><ns0:name>Google
Analytics</ns0:name></ns0:author><ns0:link
href="https://www.google.com/analytics/feeds/data?start-index=1&max-results=5000
&sort=-ga%3Apageviews&end-date=2010-12-02&start-date=2010-12-02&metrics=ga%3Apag
eviews&ids=ga%3A13538458&dimensions=ga%3ApagePath&filters=ga%3Avisits%3E0"
rel="self" type="application/atom+xml" /><ns2:endDate
xmlns:ns2="http://schemas.google.com/analytics/2009">2010-12-02</ns2:endDate><ns
2:totalResults
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">0</ns2:totalResults><ns2:sta
rtDate
xmlns:ns2="http://schemas.google.com/analytics/2009">2010-12-02</ns2:startDate><
ns2:itemsPerPage
xmlns:ns2="http://a9.com/-/spec/opensearchrss/1.0/">5000</ns2:itemsPerPage></ns0
:feed>
Thread2: result object = {}
Thread2 get data from queue: ['ga:11029900', 'st-interior.ru', False]
...
I think that's enough
Original comment by pikhovkin@gmail.com
on 3 Dec 2010 at 3:38
I can send the source code of the script
Original comment by pikhovkin@gmail.com
on 3 Dec 2010 at 3:46
The library is not written to be thread safe, and would require substantial
work to be made thread safe.
Original comment by joe.gregorio@gmail.com
on 24 Jan 2011 at 6:00
Original issue reported on code.google.com by
pikhovkin@gmail.com
on 3 Dec 2010 at 12:31