pyfa-org / Phobos

Script for getting full data dump from EVE Online client
GNU General Public License v3.0
31 stars 9 forks source link

Can't dump Singularity database #2

Closed Artefact2 closed 11 years ago

Artefact2 commented 11 years ago

This is happening for me, with Singularity build 595908 and using the singularity branch.

python2 dumpToJson.py -e ~/.eve/clients/singularity -c ~/.eve/settings/singularity/0/cache -s Singularity -j ~/Documents/EVE/json 

failed to process schematics
failed to process ramcompletedstatuses
failed to process invtypematerials
failed to process ramaltypesdetailpergroup
failed to process constellations
failed to process schematicsByPin
failed to process invgroups
processing sounds
failed to process sounds
failed to process ramaltypesdetailpercategory
failed to process eveowners
failed to process shiptypes
failed to process ramaltypes
failed to process groupsByCategories
processing icons
failed to process icons
failed to process invcontrabandTypesByFaction
failed to process solarsystems
failed to process schematicstypemap
failed to process regions
failed to process invreactiontypes
failed to process evelocations
failed to process invmetatypesByTypeID
failed to process certificates
failed to process schematicsByType
failed to process typesByMarketGroups
failed to process factions
failed to process schematicspinmap
failed to process typesByGroups
failed to process invcategories
failed to process corptickernames
failed to process ramtyperequirements
failed to process dgmeffects
failed to process fsdTypeOverrides
failed to process invmetagroups
failed to process invmetatypes
failed to process dgmexpressions
failed to process staoperationtypes
failed to process npccorporations
failed to process dgmunits
failed to process nebulas
failed to process ramactivities
processing graphics
failed to process graphics
failed to process invtypes
failed to process fsdPlanetAttributes
failed to process dgmtypeattribs
failed to process billtypes
failed to process invbptypes
failed to process mapcelestialdescriptions
failed to process invcontrabandTypesByType
failed to process dgmattribs
failed to process locationwormholeclasses
failed to process stations
failed to process certificaterelationships
failed to process dgmtypeeffects
Traceback (most recent call last):
  File "dumpToJson.py", line 109, in <module>
    for service, call in discoverSvc(eve):
  File "/home/romain/Projects/phobos/phobos/remoteSvc.py", line 33, in discover
    info, data = blue.marshal.Load(_readfile(filename))
UnmarshalError: find_global failed to resolve: carbon.common.script.net.objectCaching.CachedMethodCallResult
Exception TypeError: 'expected string or Unicode object, NoneType found' in <module 'threading' from '/usr/lib/python2.7/threading.pyc'> ignored
DarkFenX commented 11 years ago

This is not phobos' issue. CCP moved alot of stuff around which broke reverence. While Entity is away and cannot write a proper fix, i've wrote ugly hack to work around it:

diff --git a/src/blue/marshal.c b/src/blue/marshal.c
index 43e1b2b..dd5b5d2 100644
--- a/src/blue/marshal.c
+++ b/src/blue/marshal.c
@@ -221,6 +221,17 @@ find_global(PyObject *pyname)

  name = PyString_AS_STRING(pyname);

+ printf("find_global, name to find: %s\n", name);
+ if (strcmp(name, "carbon.common.script.sys.crowset.CRowset") == 0) strcpy(name, "dbutil.CRowset");
+ if (strcmp(name, "carbon.common.script.net.objectCaching.CachedMethodCallResult") == 0) strcpy(name, "objectCaching.CachedMethodCallResult");
+ if (strcmp(name, "carbon.common.script.net.cachedObject.CachedObject") == 0) strcpy(name, "util.CachedObject");
+ if (strcmp(name, "eve.common.script.sys.rowset.RowList") == 0) strcpy(name, "dbutil.RowList");
+ if (strcmp(name, "carbon.common.script.sys.crowset.CFilterRowset") == 0) strcpy(name, "dbutil.CFilterRowset");
+ if (strcmp(name, "carbon.common.script.sys.crowset.CIndexedRowset") == 0) strcpy(name, "dbutil.CIndexedRowset");
+ if (strcmp(name, "eve.common.script.sys.rowset.RowDict") == 0) strcpy(name, "dbutil.RowDict");
+ if (strcmp(name, "eve.common.script.sys.eveCfg.RamActivityVirtualColumn") == 0) strcpy(name, "util.RamActivityVirtualColumn");
+ printf("find_global, modified name: %s\n", name);
+
  dot = strchr(name, '.');
  if(dot)
  {

Note that it doesn't fix all the tables (namely, tables stored as FSD), but most of those i'm interested in are dumped properly.