search

cbrownlie / google-sites-liberation

Automatically exported from code.google.com/p/google-sites-liberation
0 stars 0 forks source link

Only Exports 10% of Site #20

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
I am trying to export a rather large site (around 1000 pages, many
attachments) and find that the tool only downloads around 50 pages. When I
then import the backup to a Google Apps domain, I'm missing the majority of
pages, including the Home page.

Using 1.0.1 on Windows 7. The problem has been replicated as I've taken
multiple exports and tried multiple imports.

Original issue reported on code.google.com by jonathan...@gmail.com on 7 Oct 2009 at 3:35

GoogleCodeExporter commented 8 years ago 
Are any error messages reported on the java console?

Original comment by gk5...@kickstyle.net on 8 Oct 2009 at 8:23

GoogleCodeExporter commented 8 years ago 
No errors that I see - the tool appears to run fine. I don't have a Java 
debugger
running though - just the JAR file executing. I'll see if I can get a more 
verbose
console output running in tandem and try the import/export again.

Original comment by jonathan...@gmail.com on 8 Oct 2009 at 9:25

GoogleCodeExporter commented 8 years ago 
Hey Jonathan,

Seems I too have a similar problem.

I am an IT Admin for a school's Google Apps site (we have an Education Edition).
Initially when I ran this tool, I couldn't edit my Google Site parallel to 
that! It
gave me "undefined" or "Unable to save the page at this time, please try again
later." errors. Having said this, all other Google services appeared normal!!

I then 'copied' my site into a new folder within my Google Site domain path to
separate out the above issue and then tried to export my site (a few hundred 
pages).
Unfortunately, all the .jar file (simple double click execution as you 
mentioned)
does is exports a few folders (out of the overall hundreds) on my hard disk but 
not
as per my Google Site structure. Furthermore, these folders only have an 
index.html
file each and nothing else :-/

Can you please post if you've got around this export problem and had your 
entire site
exported? Or if anyone can suggest something here, I would be grateful.

Cheers,
Bhavesh

Original comment by bhavesh....@gmail.com on 10 Oct 2009 at 5:51

GoogleCodeExporter commented 8 years ago 
It sounds like there are some errors somewhere, but the logging information 
isn't very 
easily found.  As a first step, I'll add something that puts the logging 
information into a 
file.  That should help with the bug reporting.

Original comment by gk5...@kickstyle.net on 12 Oct 2009 at 3:37

GoogleCodeExporter commented 8 years ago 
We are having the same problem here, in all cases, the process stops in only a 
seccion 
where we publish files.  First I thought it was a matter of time, but after a 
couple of 
hours, there was no progress.

Original comment by victorpe...@gmail.com on 19 Oct 2009 at 4:11

GoogleCodeExporter commented 8 years ago 
Unfortunately I did not find a work-around to this issue and don't have the 
time to
troubleshoot it. I did find something better though to do my migration, and have
outlined the steps here:
http://jonwestfall.com/2009/10/migrate-a-google-sites-to-a-google-apps-installat
ion/

Original comment by jonathan...@gmail.com on 21 Oct 2009 at 7:14

GoogleCodeExporter commented 8 years ago 
Thanks for the suggetion, however, we are insterested on downloading our sites 
to have 
a backup and to be able to make changes faster than editing it throught the web 
interface.  We hope they can solve the problem ASAP

Original comment by victorpe...@gmail.com on 21 Oct 2009 at 7:51

GoogleCodeExporter commented 8 years ago 
For quicker turnaround on this issue, please enable the Java console 
(http://java.com/en/download/help/javaconsole.xml) and attach the output while 
running the tool (or send it to me directly if you don't want to post the 
information).  
This will give us the information necessary to figure out why your sites aren't 
downloading.

Original comment by gk5...@kickstyle.net on 21 Oct 2009 at 8:02

GoogleCodeExporter commented 8 years ago 
Hi, we have activated the java console but it doesn't show the output while 
running the 
tool.  Is there any other way we can get this information, so we can post the 
informaction?

Original comment by victorpe...@gmail.com on 16 Nov 2009 at 4:11

GoogleCodeExporter commented 8 years ago 
Another bug report has indicated that the latest updates to Google Sites has 
fixed the 
problem.  Please try your exports again and see if it works.

Original comment by gk5...@kickstyle.net on 20 Nov 2009 at 3:52

GoogleCodeExporter commented 8 years ago 
The problem persists, the GSL downloads some files, but then, it stops (see 
attachment)

Original comment by victorpe...@gmail.com on 20 Nov 2009 at 7:22

Attachments:

GoogleCodeExporter commented 8 years ago 
I have the same issue using Mac OS X 5.8.

I am not sure how to determine my exact java config (anyone?) and do not know 
where
to find any logs that may have been generated.

Please explain how I can help trouble shoot this issue.

I have included a screenshot...

Original comment by teleserv...@gmail.com on 6 Jan 2010 at 1:41

Attachments:

GoogleCodeExporter commented 8 years ago 
This error is reproducible on MacOSX and Ubunutu.  Below is the output from the
terminal in Ubuntu...

$ java -jar google-sites-liberation-1.0.1.jar 
Exception in thread "Thread-2" java.lang.NullPointerException at
com.google.gdata.util.common.base.Preconditions.checkNotNull(Preconditions.java:
187)
at com.google.sites.liberation.util.XmlElement.addText(XmlElement.java:76) at
com.google.sites.liberation.export.HistoryExporterImpl.exportHistory(HistoryExpo
rterImpl.java:77)
at
com.google.sites.liberation.export.RevisionsExporterImpl.exportRevisions(Revisio
nsExporterImpl.java:100)
at
com.google.sites.liberation.export.SiteExporterImpl.exportSite(SiteExporterImpl.
java:133)
at
com.google.sites.liberation.util.GuiMain$ImportExportRunnable.run(GuiMain.java:2
79)
at java.lang.Thread.run(Thread.java:619)

Original comment by nelson.matt.c on 21 Jan 2010 at 9:05

GoogleCodeExporter commented 8 years ago 
In my case, this error is reproducible on Windows XP, on 3 different networks, 
on 3 
different computers.

Original comment by victorpe...@gmail.com on 21 Jan 2010 at 9:34

GoogleCodeExporter commented 8 years ago 
Reproducible on a Win XP machine. It retrieves the entries okay and then starts
grabbing the pages. And then at a seemingly random spot it dies. I don't know 
if the
last page it lists is the one it dies on but it is a different page each time I 
tried it.

C:\>java -jar google-sites-liberation-1.0.1.jar
Exception in thread "Thread-4" java.lang.NullPointerException: text
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:203)
        at
com.google.sites.liberation.renderers.RendererUtils.getHyperLink(RendererUtils.j
ava:140)
        at
com.google.sites.liberation.renderers.RendererUtils.getAuthorElement(RendererUti
ls.java:71)
        at
com.google.sites.liberation.renderers.ContentRendererImpl.renderContent(ContentR
endererImpl.java:39)
        at
com.google.sites.liberation.export.PageExporterImpl.exportPage(PageExporterImpl.
java:123)
        at
com.google.sites.liberation.export.SiteExporterImpl.exportPage(SiteExporterImpl.
java:158)
        at
com.google.sites.liberation.export.SiteExporterImpl.exportSite(SiteExporterImpl.
java:131)
        at
com.google.sites.liberation.util.GuiMain$ImportExportRunnable.run(GuiMain.java:2
79)
        at java.lang.Thread.run(Unknown Source)

Original comment by XavierBlak@gmail.com on 22 Jan 2010 at 7:49

GoogleCodeExporter commented 8 years ago 
I dont know if it's related but, after the recent update to Google Sites Api, I 
could 
download 1 site, I'm trying with another one that is larger the the first one.

Original comment by victorpe...@gmail.com on 27 Jan 2010 at 3:13

GoogleCodeExporter commented 8 years ago 
I just tried again from MacBook Pro OS X 10.5.8:

$ java -jar google-sites-liberation-1.0.1.jar 
Jan 27, 2010 8:21:11 AM com.google.sites.liberation.export.InMemoryEntryStore 
addEntry
WARNING: All non-page entries must have a parent!
Exception in thread "Thread-2" java.lang.NullPointerException: text
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:203)
    at
com.google.sites.liberation.renderers.RendererUtils.getHyperLink(RendererUtils.j
ava:140)
    at
com.google.sites.liberation.renderers.RendererUtils.getAuthorElement(RendererUti
ls.java:71)
    at
com.google.sites.liberation.renderers.ContentRendererImpl.renderContent(ContentR
endererImpl.java:39)
    at
com.google.sites.liberation.export.PageExporterImpl.exportPage(PageExporterImpl.
java:123)
    at
com.google.sites.liberation.export.SiteExporterImpl.exportPage(SiteExporterImpl.
java:158)
    at
com.google.sites.liberation.export.SiteExporterImpl.exportSite(SiteExporterImpl.
java:131)
    at com.google.sites.liberation.util.GuiMain$ImportExportRunnable.run(GuiMain.java:279)
    at java.lang.Thread.run(Thread.java:613)

SEE my previoiusly posted screenshot (it about the same)

This is a potentially invaluable tool. Anyone ever going to deal with this?...

Original comment by teleserv...@gmail.com on 27 Jan 2010 at 4:30

GoogleCodeExporter commented 8 years ago 
4 months and no news... the issue persists besides the google sites api 
update...

Original comment by victorpe...@gmail.com on 24 Feb 2010 at 8:28

GoogleCodeExporter commented 8 years ago 
I got it...
The problem is that the page that stops de export was edited by a user that has 
been deleted from out google 
apps account.
When you try to see the revision history, it says that the page was edited by 
an "unkwon user".
That must be it.

Can anyone edit the code in
com.google.sites.liberation.renderers.RendererUtils.getAuthorElement(RendererUti
ls.java:72)
and protect the line...

/**
   * Creates a new hCard element for the given entry.
   */
  static XmlElement getAuthorElement(BaseContentEntry<?> entry) {
    checkNotNull(entry);
    XmlElement element = new XmlElement("span");
    element.setAttribute("class", "author");
    XmlElement vcard = new XmlElement("span");
    vcard.setAttribute("class", "vcard");
    Person author = entry.getAuthors().get(0);
    String name = author.getName();
    String email = author.getEmail();
    if (name == null) {
     if (email == null) { // THIS LINE IS NEW
      XmlElement link = getHyperLink("mailto:", "unknown author");  // THIS LINE IS NEW
     } else {  // THIS LINE IS NEW
      XmlElement link = getHyperLink("mailto:" + email, email);
     }  // THIS LINE IS NEW
      link.setAttribute("class", "email");
      vcard.addElement(link);
    } else {
      XmlElement link = getHyperLink("mailto:" + email, name);
      link.setAttribute("class", "fn");
      vcard.addElement(link);
    }
    return element.addElement(vcard);
  }

Current progress: 0%.
Exporting page: Question?rio de Permiss?es do Sistema Inteq.
Exception in thread "main" java.lang.NullPointerException: text
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:203)
    at com.google.sites.liberation.renderers.RendererUtils.getHyperLink(RendererUtils.java:141)
    at com.google.sites.liberation.renderers.RendererUtils.getAuthorElement(RendererUtils.java:72)
    at 
com.google.sites.liberation.renderers.ContentRendererImpl.renderContent(ContentR
endererImpl.java:39)
    at com.google.sites.liberation.export.PageExporterImpl.exportPage(PageExporterImpl.java:123)
    at com.google.sites.liberation.export.SiteExporterImpl.exportPage(SiteExporterImpl.java:158)
    at com.google.sites.liberation.export.SiteExporterImpl.exportSite(SiteExporterImpl.java:131)
    at com.google.sites.liberation.export.Main.doMain(Main.java:81)
    at com.google.sites.liberation.export.Main.main(Main.java:97)

Original comment by joseluis...@gmail.com on 29 Apr 2010 at 9:45

GoogleCodeExporter commented 8 years ago 
build or fill business technology gaps based on your specific vertical, 
platform, target capability development and product direction. Cloud Computing 
was started from a non-traditional IT player.  Amazon.com, the famous web based 
bookseller, has changed the traditional computer hosting model by offering 
hosted computers
<a href="http://stratogic.com/">Hosting</a>

Original comment by stratogi...@gmail.com on 16 Nov 2010 at 10:28

GoogleCodeExporter commented 8 years ago 
Hi All. If this issue was resolved for any of you, kindly let me know.

Original comment by vikash.k...@gmail.com on 16 Nov 2010 at 6:22

GoogleCodeExporter commented 8 years ago 
This useless program is still useless.  I just tried it again (MBP 10.5.8).

All I get is a full barber-pole progress bar and a disabled "Done" button.

At one point it did do partial downloads - again, useless.

I think it should be pulled and stop wasting people's time.

Here is a command line run:

java -cp google-sites-liberation-1.0.3.jar \
>         com.google.sites.liberation.export.Main \
>         -h sites.google.com \
>         -d decdynamics.com \
>         -w team \
>         -u USER \
>         -p PASSWORD \
>         -f team/ \
>         -r
Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version 
number in .class file
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:676)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:317)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:375)
    at com.google.sites.liberation.export.Main.doMain(Main.java:67)
    at com.google.sites.liberation.export.Main.main(Main.java:97)
~/Desktop/google liberation  java -version
java version "1.5.0_24"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_24-b02-357-9M3165)
Java HotSpot(TM) Client VM (build 1.5.0_24-149, mixed mode, sharing)

Original comment by teleserv...@gmail.com on 16 Nov 2010 at 11:14

GoogleCodeExporter commented 8 years ago 
Hi, I have exact same problem as reported in comment #19. Here is my stacktrace:

WARNING: All entries should have a unique non-null id!
Exception in thread "Thread-2" java.lang.NullPointerException: text
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:203)
    at com.google.sites.liberation.renderers.RendererUtils.getHyperLink(RendererUtils.java:141)
    at com.google.sites.liberation.renderers.RendererUtils.getAuthorElement(RendererUtils.java:72)
    at com.google.sites.liberation.renderers.ContentRendererImpl.renderContent(ContentRendererImpl.java:39)
    at com.google.sites.liberation.export.PageExporterImpl.exportPage(PageExporterImpl.java:123)
    at com.google.sites.liberation.export.SiteExporterImpl.exportPage(SiteExporterImpl.java:158)
    at com.google.sites.liberation.export.SiteExporterImpl.exportSite(SiteExporterImpl.java:131)
    at com.google.sites.liberation.util.GuiMain$ImportExportRunnable.run(GuiMain.java:279)
    at java.lang.Thread.run(Thread.java:680)

Any workarounds? (I can't do the export, it always crashes at the same point)

Original comment by michal.s...@gmail.com on 4 Feb 2011 at 1:41

GoogleCodeExporter commented 8 years ago 
Why no reply from google ?
Lets it dies without any fix, why public it to customer ?

Original comment by onebit.h...@gmail.com on 2 Jun 2011 at 11:06

GoogleCodeExporter commented 8 years ago 
It worked for me most of the time, the only trouble seemed to be network 
connection problems.

But today this error "All entries should have a unique non-null id!" keeps 
showing up permanently now - in a site which did not change from the last 
successful backup yesterday!  Seems to me that there is something broken within 
sites, and this triggers this error such that one can no more backup sites.  
However according to the app dashboard everything is shown "OK" on sites, which 
I doubt, because I checked with two different ISPs with different routes to 
Google on two different machines, both show the exact same behavior, so the 
only probable common error source is Google.

I even updated sites-liberation.jar to 1.0.4, no help.

The error message is non-helpful and there certainly is missing a fix like a 
retry feature, which skips what was done properly and only retries what failed. 
 A complete run currently only backups portions of a site (at least here), and 
you cannot be sure that N partial backups done really cover all data in a site.

Again to stress it, I have an automated daily backup, and it ran fine the last, 
say, 100 days.  So, please, Google, fix that.  Thanks.

Here is one example:

Retrieved 1120 entries.
Jul 1, 2011 12:58:31 PM 
com.google.sites.liberation.export.ContinuousContentFeed$FeedIterator 
catchException
WARNING: Error retrieving response from query.
com.google.gdata.util.ServiceException: Internal Server Error
Internal Error

        at com.google.gdata.client.http.HttpGDataRequest.handleErrorResponse(HttpGDataRequest.java:588)
        at com.google.gdata.client.http.GoogleGDataRequest.handleErrorResponse(GoogleGDataRequest.java:543)
        at com.google.gdata.client.http.HttpGDataRequest.checkResponse(HttpGDataRequest.java:536)
        at com.google.gdata.client.http.HttpGDataRequest.execute(HttpGDataRequest.java:515)
        at com.google.gdata.client.http.GoogleGDataRequest.execute(GoogleGDataRequest.java:515)
        at com.google.gdata.client.Service.getFeed(Service.java:1053)
        at com.google.gdata.client.Service.getFeed(Service.java:995)
        at com.google.gdata.client.GoogleService.getFeed(GoogleService.java:662)
        at com.google.gdata.client.Service.getFeed(Service.java:952)
        at com.google.sites.liberation.util.EntryProviderImpl.getEntries(EntryProviderImpl.java:45)
        at com.google.sites.liberation.export.ContinuousContentFeed$FeedIterator.getEntries(ContinuousContentFeed.java:141)
        at com.google.sites.liberation.export.ContinuousContentFeed$FeedIterator.computeNext(ContinuousContentFeed.java:115)
        at com.google.sites.liberation.export.ContinuousContentFeed$FeedIterator.computeNext(ContinuousContentFeed.java:95)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:135)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:130)
        at com.google.sites.liberation.export.SiteExporterImpl.exportSite(SiteExporterImpl.java:101)
        at com.google.sites.liberation.export.Main.doMain(Main.java:81)
        at com.google.sites.liberation.export.Main.main(Main.java:97)
Jul 1, 2011 12:59:14 PM com.google.sites.liberation.export.InMemoryEntryStore 
addEntry
WARNING: All entries should have a unique non-null id!
Retrieved 1140 entries.

Original comment by valentin...@googlemail.com on 1 Jul 2011 at 11:42

GoogleCodeExporter commented 8 years ago 
@ #6:

Thank you SO much.  Your method worked flawlessly and instantly, completely 
bypassing this tool.

Original comment by ccat...@gmail.com on 14 Oct 2011 at 6:47

GoogleCodeExporter commented 8 years ago 
I had same problem too. Some pages were exported by this tool, not all of them. 
I noticed liberation tool exports recent posts only. So, I modified old posts 
little, for example, just click the modification button and save(I did 
modification more easier than this in the site management, but I can't explain 
this well...). Then, I could export all the posts.

Original comment by star1...@gmail.com on 17 Feb 2015 at 9:06