no results

[GoogleCodeExporter]

Hello,

I tested your code and with your lookup example i dont get a positive result 
with the second url. Also the test url provided by google is not working.

Is it normal to get a lot of DISCARDED CHUNKNUM in the updater? See for a live 
version http://testsite.tcpiputils.com/listupdater.php.

Regards, Bill

Original issue reported on code.google.com by i...@tcpiputils.com on 2 Jul 2010 at 10:40

[GoogleCodeExporter]

Am investigating now, you shouldn't be getting those, basically a chunk should 
contain either: S or A headers, it discards them if for some reason the header 
isn't either. Just waiting for the timeout to finish so I can pull up the error 
data again.

Original comment by sam%beav...@gtempaccount.com on 5 Jul 2010 at 6:59

• Changed state: Investigating

[GoogleCodeExporter]

Could you try the class here:
http://phpgsb.googlecode.com/svn/branches/debug/phpgsb.class.php
It should dump any chunkdata into .debug files
Could you reply with a few of those attached please?

Original comment by sam%beav...@gtempaccount.com on 5 Jul 2010 at 7:49

• Changed state: Awaiting

[GoogleCodeExporter]

I tried your class. You can access the debug files here:
http://testsite.tcpiputils.com/DUMP0.debug its till 17.

Original comment by i...@tcpiputils.com on 5 Jul 2010 at 8:26

[GoogleCodeExporter]

Original comment by sam%beav...@gtempaccount.com on 5 Jul 2010 at 8:30

• Changed state: Investigating

[GoogleCodeExporter]

i see you are using the mb_ functions. Perhaps you need to know i have a 
default encoding set to utf8.

Original comment by i...@tcpiputils.com on 5 Jul 2010 at 8:31

[GoogleCodeExporter]

So do I, I'll try processing some of those chunks on my server and I'll see if 
it manages to process them.

Original comment by sam%beav...@gtempaccount.com on 5 Jul 2010 at 8:33

[GoogleCodeExporter]

Odd it processed that first one fine, in your listupdater.php can you find:
$phpgsb->runUpdate();
and replace with:
$chunkinfo = file_get_contents("DUMP0.debug");
print_r($phpgsb->processChunks($chunkinfo));

And let me know what it outputs?

Original comment by sam%beav...@gtempaccount.com on 5 Jul 2010 at 9:03

• Changed state: Awaiting

[GoogleCodeExporter]

you can see the output here:
http://testsite.tcpiputils.com/listupdater.php

Original comment by i...@tcpiputils.com on 5 Jul 2010 at 9:15

[GoogleCodeExporter]

Thats very odd, we're using the same code, the same debug file and yet we get 
different outputs.. hmm (that error at the top of yours for some reason is 
there because the reading of the data seems to go awry at the end of the first 
chunk.) 

I'll do some more tests tommorow and will likely have some more code for you to 
try! 
You could try removing the mb_ inside processchunks function and see if that 
makes a difference. It'll help eliminate that as a source of the problem.

Original comment by sam%beav...@gtempaccount.com on 5 Jul 2010 at 9:30

• Changed state: Investigating

[GoogleCodeExporter]

i changed the mb_ function and now i dont get any error:

http://testsite.tcpiputils.com/listupdater1.php

Original comment by i...@tcpiputils.com on 5 Jul 2010 at 9:35

[GoogleCodeExporter]

Hmm very odd, I originally used the functions as a precaution though it should 
be okay with the regular substr, I'll get it replaced in the source. Thanks for 
flagging this up! You might want to clear the database and start fresh now to 
avoid any corruption. Is it all working correctly now?

Original comment by sam%beav...@gtempaccount.com on 5 Jul 2010 at 9:44

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

yes i deleted the comment because i was not finished yet...

but i still get two false results with your lookup example:

http://testsite.tcpiputils.com/lookup.php

Original comment by i...@tcpiputils.com on 5 Jul 2010 at 10:36

[GoogleCodeExporter]

The database could take around 24 hours to fully populate the database (you 
have to keep running the updater, I'd put it on a cron running every minute [it 
won't actually update every minute but will ping the timeout file]). Google 
don't provide all the information in one go you have to keep updating and 
eventually you get the majority of the chunks.

Original comment by sam%beav...@gtempaccount.com on 5 Jul 2010 at 10:41

• Changed state: Awaiting

[GoogleCodeExporter]

There are still problems...only the *index tables get populated.

I did some debug and i see for example the Insert hostkeys index query is 
failing with the error Field 'FullHash' doesn't have a default value.

try this one on your machine:
 INSERT INTO `phpgsb`.`goog-malware-shavar-s-hosts` (`ID`,`Hostkey`,`Chunknum`,`Count`) VALUES ('','cfa8d53a','33961','00')

Original comment by i...@tcpiputils.com on 6 Jul 2010 at 6:26

[GoogleCodeExporter]

i see the problem..the field fullhash has the property Not Null and there is 
not a default value. If i change it to a default value i get a new error:

Incorrect integer value: '' for column 'ID' at row 1

Original comment by i...@tcpiputils.com on 6 Jul 2010 at 6:35

[GoogleCodeExporter]

Could you try using this version:
http://phpgsb.googlecode.com/svn/trunk/phpgsb.class.php
And let me know the results.

Original comment by sam%beav...@gtempaccount.com on 6 Jul 2010 at 7:42

[GoogleCodeExporter]

the FullHash is solved..now i get

INSERTED goog-malware-shavar SUB HOST KEYS Incorrect integer value: '' for 
column 'ID' at row 1...

Original comment by i...@tcpiputils.com on 6 Jul 2010 at 7:57

[GoogleCodeExporter]

That shouldn't be happening... can you check that the ID field is set to 
Auto-Increment in the database?

Original comment by sam%beav...@gtempaccount.com on 6 Jul 2010 at 8:01

[GoogleCodeExporter]

yes it is AI. You have to remove the column ID from Your insert statement 
because it is automaticly populated by mysql.

Original comment by i...@tcpiputils.com on 6 Jul 2010 at 8:14

[GoogleCodeExporter]

Hmm I've never had a problem with that before, I've removed the ID column in 
this version:
http://phpgsb.googlecode.com/svn/trunk/phpgsb.class.php
Does that prevent the problem?

Original comment by sam%beav...@gtempaccount.com on 6 Jul 2010 at 8:17

[GoogleCodeExporter]

it seems to work now..i will perform some test to be sure..thanx

Original comment by i...@tcpiputils.com on 6 Jul 2010 at 8:50

[GoogleCodeExporter]

Could you confirm if its now working now?

Original comment by sam%beav...@gtempaccount.com on 7 Jul 2010 at 4:00

[GoogleCodeExporter]

yes it works but i get a lot and lot notices:

PHP Notice:  Undefined index: PREFIX in phpgsb.class.php on line 307

Can You fix this?

Original comment by i...@tcpiputils.com on 7 Jul 2010 at 6:02

[GoogleCodeExporter]

For some reason you're setup must be using error_reporting(E_ALL);
In listupdater.php and lookup.php the following to the top (after <?php ):

error_reporting(E_ALL ^ E_NOTICE);

At a later date I'll rework the code to check for existances of variables more. 
(They're not actually errors)

Original comment by sam%beav...@gtempaccount.com on 7 Jul 2010 at 6:06

[GoogleCodeExporter]

Original comment by sam%beav...@gtempaccount.com on 7 Jul 2010 at 7:44

• Changed state: Verified