Closed GoogleCodeExporter closed 9 years ago
On a different server, I cannot reproduce this error.
Perhaps there was still an old database on the server? But then why was it not
dropped by the setup-script?
I've marked this issue as 'Started' and 'medium'. Any further tests would be
appreciated.
FWieP
Original comment by fwp...@gmail.com
on 21 Feb 2012 at 5:49
I tried it and was not able to replicate the problem.
I am leaning towards an old database left behind. r292 allowed for custom DB
names, so possibly there was a reference to the old DB left behind
Original comment by mranimos...@gmail.com
on 22 Feb 2012 at 4:56
dont know if it helps, but I am running Apache 2.2.17 with PHP 5.3.5 and MySql
5.5.8, installed through XAMPP 1.7.4 on Windows 7
Original comment by mranimos...@gmail.com
on 22 Feb 2012 at 4:59
I'm at work now, and have reproduced the problem. It's a virtual-pc running
Ubuntu Lucid, Apache 2.2.14 and MySQL 5.1.41.
The database is being dropped and recreated nicely.
When I start the Process-XML on the command line, the progress indicator rises
above 100%, to about 170%.
After that, I inspect the inserted records (models) with PhpMyAdmin. I see that
up until Sonya M (id = 77) all is well, but then the AUTO-INCREMENT jumps
forward. This indicates the UNIQUE errors during import.
I validate and use the latest XML-file, use the latest setup_data.php from SVN
and have verified that the database is clean before importing.
I suspect setup_data.php is looping more than it should.
FWieP
Original comment by fwp...@gmail.com
on 22 Feb 2012 at 11:50
this was added to setup_data recently, cannot find the revision number, but i
know it was recent
// Provide a one-model-only import for impatient developers
if($Model2Process->getID() && $ModelID && $Model2Process->getID() !== $ModelID)
{ continue; }
try removing it and then see if the problem is re-creatable
Original comment by mranimos...@gmail.com
on 22 Feb 2012 at 10:06
Tried your suggestion, to no avail.
Tried completely removing the cd-folder and doing a new check-out of r328.
Setup, import, BOOM: unique violations after ID=77.
Curious detail: Today a new model (Abrora H) was added to the XML, thus
shifting Sonya M up by 1, now the import still BOOMs at ID=77, being Sofiya V...
This is getting weirder by the minute... I'll give it some more tries, but I'm
beginning to suspect a messed-up PHP-MySQL-combination...
FWieP
Original comment by fwp...@gmail.com
on 23 Feb 2012 at 11:30
Looked for differences between this server and my dev-machine. Could (one of)
these be causing problems?
- Core PHP: 5.3.2-1ubuntu4.14
mine: 5.3.6-13ubuntu3.6
- LibXML: 2.7.6
mine: 2.7.8
- Mysql client: 5.1.41
mine: 5.1.58
- Mysqli client: 5.1.41
mine: 5.1.58
- SimpleXML: Revision: 293036
mine: Revision: 308262
FWieP
Original comment by fwp...@gmail.com
on 23 Feb 2012 at 11:45
will try a new checkout and install again after work
Original comment by mranimos...@gmail.com
on 23 Feb 2012 at 11:59
checked out r329, processed XML and no errors showed. Dropped the database and
setup again, this time processing the XML on the CLI, went to 100% with no
errors. As you can see below, most of my versions are below yours, so I don't
know if that would be the problem since it obviously worked fine for awhile I
assume
- Server: Apache/2.2.17 (Win32) on Windows 7 (installed via XAMPP 1.7.4)
- Core PHP: 5.3.5
yours: 5.3.6-13ubuntu3.6
- LibXML: 2.7.7
yours: 2.7.8
- Mysql client: 5.0.7-dev - 091210 - Revision: 304625
yours: 5.1.58
- Mysqli client: 5.0.7-dev - 091210 - Revision: 304625
yours: 5.1.58
- SimpleXML: Revision: 302715
yours: Revision: 308262
Original comment by mranimos...@gmail.com
on 24 Feb 2012 at 9:12
More testing on this very weird virtual-pc... When I comment out the last 25%
of the XML, so that only ~75 models will be imported, that's exactly what
happens. Without errors.
When I put ONE more model in the XML, the CLI-progress indicator loops from
0-200%. As expected, the last AUTO-INCREMENT has junped forward.
First I suspected a problem in the InnoDB handling of NULLable columns in
UNIQUE-constraints. So, I modified the Model-class to give Lena and Oksana a
dummy lastname. No go...
Remark: There are the correct number of 99 models present in the database. So
there definitely is something to these NULLable lastnames...
Original comment by fwp...@gmail.com
on 24 Feb 2012 at 11:40
speaking of Lena, who is she, i have yet to see anything on her, same goes for
Alina N
Original comment by mranimos...@gmail.com
on 24 Feb 2012 at 11:53
Now with provable numbers. I ran three import runs, each starting with a clean
DB.
1) Commented out the last models, so that from AbroraH -> SilviyaR would be
imported. This yielded 100% and 76 models. All is well.
2) De-commented one more model. The range became AbroraH -> SofiyaV. This
yielded 200% and 79 records. Lena and Oksana were added twice! Both these sets
had a jumped model_id.
3) Ran the same run (AbroraH -> SofiyaV) with dummy lastnames for Lena and
Oksana. This gave me 77 records but still 200%. One abbreviation that would fit
my reaction would be w.t.f..?
I have no idea what is going wrong here, but I at least have some data to
research, look up, ponder about.
FWieP
Original comment by fwp...@gmail.com
on 24 Feb 2012 at 12:08
Alina N appeared some months ago on the CD-site, with one sample image. Never
seen a set of her.
Lena is the fourth girl in the SP_4Dress special set. When I learned her name,
I thought it right to add her to the collection. No sets of her, either.
FWieP
Original comment by fwp...@gmail.com
on 24 Feb 2012 at 12:10
More testing on that less than fortunate virtual-pc... Finally some progress!
Ran a new run with all source code updated to v1.6, including setup_data.php
and its XML-file. Now it runs to about 140%, then finishes. Then it spits out
the following error:
zend_mm_heap corrupted
Google searches revealed some indication to garbagecollection, but I haven't
been able to find the bug as such... At least we now know there IS a bug to be
found.
FWieP
Original comment by fwp...@gmail.com
on 27 Feb 2012 at 12:02
Removed and purged PHP, re installed PHP, no go.
Removed and purged MySQL, re installed MySQL, no go.
The zend_mm_heap error never showed up again...
Original comment by fwp...@gmail.com
on 2 Mar 2012 at 11:39
Ran another couple of runs: Remove config, re-setup (being sure that the DB was
being dropped in between), import using CLI of GUI.
First run with most up to date XML. It processed, then complains about UNIQ_SET
which is being violated for model_id=77. Later I see the following range of
model_ids:
AbroraH SofiyaV Lena Oksana SonyaM ZinaB VIP Promo Interv.
1 ..... 77 120 139 154 ...... 173 174 175 176
During this same run on the command line, I got a 'segmentation fault'.
When I comment all but the first 76 models, all is well and no errors are
thrown.
I think that Lena and Oksana are imported twice because of some NULL-check
going wrong. I'd like to get this #$&^!-76 issue fixed first.
Original comment by fwp...@gmail.com
on 6 Mar 2012 at 12:13
try adding a fake last name for Lena and Oksana, see if that works, if so then
it definitely is a bad Null-check somewhere
Original comment by mranimos...@gmail.com
on 7 Mar 2012 at 6:49
Tried your suggestion, added 'DUMMY' as last name for Lena and Oksana.
AbroraH SofiyaV SonyaM ZinaB VIP Promo Interv.
1 ..... 77 154 ...... 173 174 175 176
This is getting weirder and weirder. I'm open to any other suggestions?
Original comment by fwp...@gmail.com
on 8 Mar 2012 at 10:20
This is the list of errors, as thrown when one clicks on 'Process XML'.
Original comment by fwp...@gmail.com
on 8 Mar 2012 at 11:39
Attachments:
Dropped and set up once again, now without the NULLable column 'set_prefix' in
UNIQ_SET. Same result, same errors.
Original comment by fwp...@gmail.com
on 8 Mar 2012 at 12:06
One more run, this time with SharlottaS sets 2-33 removed: Progress!
AbroraH SonyaM Lena Oksana SundiZ ZinaB VIP Promo Interv.
1 ..... 78 121 140 156 ...... 174 175 176 177
It seems that the total amount of sets plays a part in this weird error. This
weekend I'll investigate some more, at home.
Original comment by fwp...@gmail.com
on 9 Mar 2012 at 12:26
Attachments:
don't know if you tried this yet, but try commenting out the VIP, Promotion &
Interview parts of the XML, or adding a blank last name attritube, then
re-running it becuase I noticed there is no last name attribute for those in
the XML
Original comment by mranimos...@gmail.com
on 10 Mar 2012 at 2:24
Have been debugging this issue now for 5 hours non-stop. Filled in
dummy-lastnames for Lena, Oksana and the 3 specials. The foreach-loop of all
SimpleXML Models begins anew after having processed its 78th node. In the
second run it processes model 1-101, as it should. The importing code normally
requests all Models, Sets and Dates ONCE at the start of the run.
During the second run, the first 78 models, their sets and their dates produce
the loads and loads of UNIQUE-violations. A temporary work-around would be to
add the freshly inserted Models, Sets and Dates to these arrays ($ModelsInDb,
$SetsInDb and $DatesInDb). But this not solving the problem, just hiding it.
And, this run takes ages when the first 78 models are processed a second time...
I'm going to simplify/strip the XML to a 'simple' test case that can be easily
brought to the attention of more knowledgeable people, perhaps a problem in
SimpleXML ? I have no idea...
Original comment by fwp...@gmail.com
on 10 Mar 2012 at 8:06
All right, one more hour of debugging. I'm beginning to suspect some magic
limit of SimpleXML, PHP or a combination of both.
Models are inserted nicely, they get model_id's 1-13. Sets however:
Set01 Set1239 Set1240 Set1300
1 ..... 1239 1278 .... 1338
This time the same thing happens as earlier from SofiyaV onwards, but around
the 1239th set. I still can't explain this. Will try a direct approach of 1
model with 1239 sets. See if that makes it go woozy...
Original comment by fwp...@gmail.com
on 10 Mar 2012 at 9:16
Attachments:
1239 seems to be the magic number of this error. All sets have set_ids
corresponding to their names, but an enormous load of errors is thrown because
of a double run of the XML.
Set1 Set1239
1 ..... 1239
I did a run with 1238 sets (removed last set-node from XML), and all went well,
without any errors.
Enough for today, I'm going to bed.
Original comment by fwp...@gmail.com
on 10 Mar 2012 at 9:34
Attachments:
Once more a few hours of debugging at home, and just now a re-run on that
particular virtual pc. Progress? Yes. Solution? No.
At home I determined that after the processing of model #78, the XML loop
starts over again and causes loads of DUPLICATE KEY errors. When I commented
out most of setup_data.php, it ran through flawlessly. So I de-commented one
piece of code at a time and discovered the evil line (around line 124):
$CacheImages = CacheImage::GetCacheImages(sprintf('index_id = %1$d', $Model2Process->getID()));
I changed it into
$CacheImages = array();
This did the trick! Why? I don't know. I can only summarize that if this code
runs, this machine goes frenzy after 78 models/12xx sets/13xx XML nodes. I'm
open to suggestions, and most of all a logical explanation.
Anyone, please?
Original comment by fwp...@gmail.com
on 12 Mar 2012 at 11:49
Traced it down some more, but not quite nailed it - yet.
After 78 models, the GetCacheImages causes the XML-loop to repeat itself. I
have tracked it into the $db->Select() function. There something goes wrong in
the filling of the FieldInfo array. It is later used to cast the db-values to
their corresponding datatypes (currently only int and string).
If I comment out the call to mysql_fetch_field() and set the $db-FieldInfo to
NULL, all is well and no errors are thrown. This looks promising, I'll put more
time in it, when I'm able to.
Original comment by fwp...@gmail.com
on 13 Mar 2012 at 12:52
Finally found it, and fixed it in r381.
I did not reset the $db->FieldInfo array when making a new SQL-query. This
array got bigger, and bigger, and bigger... Up to about 1024, where I think it
made PHP kick the bucket and go crazy.
When I debugged it using a conditional breakpoint, after 1230 calls to the
$db-Select, the array's count was about 1000. When I noticed this, I added a
reset statement - as I seem to have done from the beginning for the $db's
result-property.
Well, this is one for (my personal) recordbooks. I'm glad it's finally fixed
:-) Thanks for thinking along!
Original comment by fwp...@gmail.com
on 13 Mar 2012 at 7:14
Original issue reported on code.google.com by
fwp...@gmail.com
on 21 Feb 2012 at 11:50