search

AppScale / gts

AppScale is an easy-to-manage serverless platform for building and running scalable web and mobile applications on any infrastructure.
https://www.appscale.com/community/
Apache License 2.0
2.42k stars 277 forks source link

Support GAE API 1.9.x #1900

Open fmntf opened 8 years ago

fmntf commented 8 years ago

Hello,

I'm wondering if there is a plan to supported the APIs introduced in App Engine 1.9.x Current SDK is 1.9.30, and applications relying on newer APIs cannot run on AppScale. For istance, i tried AppInventor 2 without success.

Thank you, Francesco

cdonati commented 8 years ago

Hi Francesco,

We do plan on eventually supporting the 1.9.x sdk, and we are making progress towards that goal.

The good news is that you can already run AppInventor 2 on AppScale. You just need to use the 1.8.4 sdk to compile it. Here's some more detailed instructions:

Let me know if you have any issues.

MCSH commented 8 years ago

Hello,

following your instructions I got this error message:

     [java] INFO: The current progress is 85%
     [java] Jan 19, 2016 8:16:29 AM com.google.appinventor.buildserver.Execution
 execute
     [java] INFO: ____Executing /tmp/aapt4661248222253282693 package -v -f -M /t
mp/1453191291362_0.31347609593622106-0/youngandroidproject/../build/AndroidManif
est.xml -S /tmp/1453191291362_0.31347609593622106-0/youngandroidproject/../build
/res -A /tmp/1453191291362_0.31347609593622106-0/youngandroidproject/../assets -
I /tmp/android1713192223360172802.jar -F /tmp/1453191291362_0.31347609593622106-
0/youngandroidproject/../build/deploy/MIT AI2 Companion.ap_ /tmp/1453191291362_0
.31347609593622106-0/youngandroidproject/../build/libs
     [java] Jan 19, 2016 8:16:29 AM com.google.appinventor.buildserver.Execution
 execute
     [java] WARNING: ____Execution failure:
     [java] java.io.IOException: Cannot run program "/tmp/aapt466124822225328269
3": error=2, No such file or directory
     [java]     at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
     [java]     at java.lang.Runtime.exec(Runtime.java:617)
     [java]     at com.google.appinventor.buildserver.Execution.execute(Executio
n.java:111)
     [java]     at com.google.appinventor.buildserver.Compiler.runAaptPackage(Co
mpiler.java:1119)
     [java]     at com.google.appinventor.buildserver.Compiler.compile(Compiler.
java:613)
     [java]     at com.google.appinventor.buildserver.ProjectBuilder.build(Proje
ctBuilder.java:164)
     [java]     at com.google.appinventor.buildserver.Main.main(Main.java:82)
     [java] Caused by: java.io.IOException: error=2, No such file or directory
     [java]     at java.lang.UNIXProcess.forkAndExec(Native Method)
     [java]     at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
     [java]     at java.lang.ProcessImpl.start(ProcessImpl.java:130)
     [java]     at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
     [java]     ... 6 more
     [java]
     [java] Jan 19, 2016 8:16:29 AM com.google.appinventor.buildserver.Compiler
runAaptPackage
     [java] WARNING: YAIL compiler - AAPT execution failed.

BUILD FAILED
/vagrant/appinventor-sources/appinventor/build.xml:21: The following error occur
red while executing this line:
/vagrant/appinventor-sources/appinventor/buildserver/build.xml:183: Java returne
d: 1

Total time: 9 minutes 35 seconds

after trying to build with ant.

What could be the cause of this error? How can I fix this?

Thank you, Sajjad

MCSH commented 8 years ago

Update:

I was able to compile the edited appinventor-sources under my original development environment however it failed in appscale vagrant instance.

cdonati commented 8 years ago

Hi Sajjad,

When you say it failed in the appscale vagrant instance, do you mean the compilation or deploying the compiled app?

If you are having trouble getting the compiled app to work, you are welcome to send me the logs from your vagrant deployment and I will take a look at them.

MCSH commented 8 years ago

Hi Chris,

The compilation failed. The problem was I was compiling using OpenJDK and it's not compatible.

However after compiling with the correct java version and deploying, I constantly get the popup "unable to save files temporally, please try again later" and with the following log:

javax.servlet.ServletContext log: Exception while dispatching incoming RPC call
com.google.gwt.user.server.rpc.UnexpectedException: Service method 'public abstract long com.google.appinventor.shared.rpc.project.ProjectService.save2(java.lang.String,long,java.lang.String,boolean,java.lang.String) throws com.google.appinventor.shared.rpc.InvalidSessionException,com.google.appinventor.shared.rpc.BlocksTruncatedException' threw an unexpected exception: com.google.appengine.tools.cloudstorage.NonRetriableException: java.lang.RuntimeException: Server replied with 401, probably bad authentication: Request: POST https://storage.googleapis.com/app_default_bucket/1/src/appinventor/ai_sajjadheydari74/Test/Screen1.bky.2016-01-19T10:25:00+0000.backup
User-Agent: App Engine GCS Client
x-goog-resumable: start

no content

Response: 401 with 131 bytes of content
X-GUploader-UploadID: AEnB2UpVzNUE4lOaCf5RdpiC3GZowMv5FQnqNZEVGQIEkWJsI1A2K1DlnCv95CniarYiVI48NYBRPhcNXmHyAbJtQxQEmSaqlA
Content-Type: application/xml; charset=UTF-8
Content-Length: 131
WWW-Authenticate: Bearer realm="https://accounts.google.com/", error="invalid_token"
Vary: Origin
Date: Tue, 19 Jan 2016 10:27:47 GMT
Server: UploadServer
Alternate-Protocol: 443:quic,p=1
Alt-Svc: quic=":443"; ma=604800; v="30,29,28,27,26,25"
AuthenticationRequiredAuthentication required.

    at com.google.gwt.user.server.rpc.RPC.encodeResponseForFailure(RPC.java:415)
    at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:605)
    at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:333)
    at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:303)
    at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:373)
    at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166)
    at com.google.appinventor.server.OdeAuthFilter.doMyFilter(OdeAuthFilter.java:106)
    at com.google.appinventor.server.OdeAuthFilter.doFilter(OdeAuthFilter.java:78)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java:141)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.api.socket.dev.DevSocketFilter.doFilter(DevSocketFilter.java:74)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.development.ResponseRewriterFilter.doFilter(ResponseRewriterFilter.java:123)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:34)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:63)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:125)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectRequest(DevAppServerModulesFilter.java:368)
    at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectModuleRequest(DevAppServerModulesFilter.java:351)
    at com.google.appengine.tools.development.DevAppServerModulesFilter.doFilter(DevAppServerModulesFilter.java:116)
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
    at com.google.appengine.tools.development.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:97)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:485)
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
    at org.mortbay.jetty.Server.handle(Server.java:326)
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
    at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:938)
    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755)
    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: com.google.appengine.tools.cloudstorage.NonRetriableException: java.lang.RuntimeException: Server replied with 401, probably bad authentication: Request: POST https://storage.googleapis.com/app_default_bucket/1/src/appinventor/ai_sajjadheydari74/Test/Screen1.bky.2016-01-19T10:25:00+0000.backup
User-Agent: App Engine GCS Client
x-goog-resumable: start

no content

Response: 401 with 131 bytes of content
X-GUploader-UploadID: AEnB2UpVzNUE4lOaCf5RdpiC3GZowMv5FQnqNZEVGQIEkWJsI1A2K1DlnCv95CniarYiVI48NYBRPhcNXmHyAbJtQxQEmSaqlA
Content-Type: application/xml; charset=UTF-8
Content-Length: 131
WWW-Authenticate: Bearer realm="https://accounts.google.com/", error="invalid_token"
Vary: Origin
Date: Tue, 19 Jan 2016 10:27:47 GMT
Server: UploadServer
Alternate-Protocol: 443:quic,p=1
Alt-Svc: quic=":443"; ma=604800; v="30,29,28,27,26,25"
AuthenticationRequiredAuthentication required.

    at com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:120)
    at com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:166)
    at com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:156)
    at com.google.appengine.tools.cloudstorage.GcsServiceImpl.createOrReplace(GcsServiceImpl.java:70)
    at com.google.appinventor.server.storage.ObjectifyStorageIo$42.run(ObjectifyStorageIo.java:1476)
    at com.google.appinventor.server.storage.ObjectifyStorageIo.runJobWithRetries(ObjectifyStorageIo.java:2264)
    at com.google.appinventor.server.storage.ObjectifyStorageIo.uploadRawFile(ObjectifyStorageIo.java:1397)
    at com.google.appinventor.server.storage.ObjectifyStorageIo.uploadRawFile(ObjectifyStorageIo.java:1382)
    at com.google.appinventor.server.storage.ObjectifyStorageIo.uploadFile(ObjectifyStorageIo.java:1328)
    at com.google.appinventor.server.project.CommonProjectService.save2(CommonProjectService.java:268)
    at com.google.appinventor.server.ProjectServiceImpl.save2(ProjectServiceImpl.java:464)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Unknown Source)
    at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:587)
    ... 46 more
Caused by: java.lang.RuntimeException: Server replied with 401, probably bad authentication: Request: POST https://storage.googleapis.com/app_default_bucket/1/src/appinventor/ai_sajjadheydari74/Test/Screen1.bky.2016-01-19T10:25:00+0000.backup
User-Agent: App Engine GCS Client
x-goog-resumable: start

no content

Response: 401 with 131 bytes of content
X-GUploader-UploadID: AEnB2UpVzNUE4lOaCf5RdpiC3GZowMv5FQnqNZEVGQIEkWJsI1A2K1DlnCv95CniarYiVI48NYBRPhcNXmHyAbJtQxQEmSaqlA
Content-Type: application/xml; charset=UTF-8
Content-Length: 131
WWW-Authenticate: Bearer realm="https://accounts.google.com/", error="invalid_token"
Vary: Origin
Date: Tue, 19 Jan 2016 10:27:47 GMT
Server: UploadServer
Alternate-Protocol: 443:quic,p=1
Alt-Svc: quic=":443"; ma=604800; v="30,29,28,27,26,25"
AuthenticationRequiredAuthentication required.

    at com.google.appengine.tools.cloudstorage.oauth.HttpErrorHandler.createException(HttpErrorHandler.java:54)
    at com.google.appengine.tools.cloudstorage.oauth.HttpErrorHandler.error(HttpErrorHandler.java:32)
    at com.google.appengine.tools.cloudstorage.oauth.HttpErrorHandler.error(HttpErrorHandler.java:59)
    at com.google.appengine.tools.cloudstorage.oauth.OauthRawGcsService.beginObjectCreation(OauthRawGcsService.java:246)
    at com.google.appengine.tools.cloudstorage.GcsServiceImpl$1.call(GcsServiceImpl.java:74)
    at com.google.appengine.tools.cloudstorage.GcsServiceImpl$1.call(GcsServiceImpl.java:70)
    at com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:108)
    ... 62 more
cdonati commented 8 years ago

You're right about the compilation failing in an AppScale vagrant instance. I only tested building it with openjdk 7.u91, which is much newer than the version installed on Precise.

As for the GCS errors you are getting, try editing the appengine-web.xml file and changing 'use.gcs' to 'false'. After re-deploying, I did not get those errors anymore.

MCSH commented 8 years ago

Thank you. I can verify that the problem is solved.

MCSH commented 8 years ago

Hello,

It appears I'm getting other errors upon building. One was solved here however there is still another problem which I wasn't able to solve.

Upon a successful build, the files are not sent back to the url, getting a 500 error. I modified logs a little to get more information. Full buildserver log: log.txt

Some parts that seems related:

     [java] INFO: ZIPALIGN time: 0.021 seconds
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.Compiler setProgress
     [java] INFO: The current progress is 100%
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer build
     [java] INFO: Build output: ________Preparing application icon<br>________Creating animation xml<br>________Determining permissions<br>________Generating manifest file<br>________Attaching native libraries<br>________Copying native libraries<br>________Attaching component assets<br>________Compiling source files<br>(compiling appinventor/ai_sajjadheydari74/test/Screen1.yail to appinventor.ai_sajjadheydari74.test.Screen1)<br>(compiling /tmp/runtime7048472273610945046.scm to com.google.youngandroid.runtime)<br>Kawa compile time: 2.496 seconds<br>________Invoking DX<br>DX time: 6.234 seconds<br>________Invoking AAPT<br>AAPT time: 0.023 seconds<br>________Invoking ApkBuilder<br>________Signing the apk file<br>________ZipAligning the apk file<br>ZIPALIGN time: 0.021 seconds<br>Build finished in 10.685 seconds<br>
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer build
     [java] INFO: Build error output: 
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer checkMemory
     [java] INFO: Build 1 current used memory: 12063536 bytes
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer$1 run
     [java] INFO: CallbackURL: http://192.168.2.26:80/ode2/receivebuild/vb4ftxd7m12fh5urhteyazsvqiieeip5go4p3pnb32w7xna5sbi1zcpggbl3bx8xxtpfuyz4vo1dz7zdjq6s1vwmw2ru69iafami63s2bnueld2wg/build/Android
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer checkMemory
     [java] INFO: Build 1 current used memory: 14229544 bytes
     [java] null=[HTTP/1.1 500 The request to API call datastore_v3.Put() was too large.]
     [java] Date=[Wed, 24 Feb 2016 11:52:37 GMT]
     [java] Content-Length=[2542]
     [java] X-TraceUrl=[/appstats/details?time=1456314757517&type=json]
     [java] Connection=[keep-alive]
     [java] Content-Type=[text/html; charset=iso-8859-1]
     [java] Server=[nginx/1.1.19]
     [java] Cache-Control=[must-revalidate,no-cache,no-store]
     [java] <html>
     [java] <head>
     [java] <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
     [java] <title>Error 500 The request to API call datastore_v3.Put() was too large.</title>
     [java] </head>
     [java] <body><h2>HTTP ERROR 500</h2>
     [java] <p>Problem accessing /ode2/receivebuild/vb4ftxd7m12fh5urhteyazsvqiieeip5go4p3pnb32w7xna5sbi1zcpggbl3bx8xxtpfuyz4vo1dz7zdjq6s1vwmw2ru69iafami63s2bnueld2wg/build/Android. Reason:
     [java] <pre>    The request to API call datastore_v3.Put() was too large.</pre></p><h3>Caused by:</h3><pre>com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call datastore_v3.Put() was too large.
     [java]     at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.callInternal(Unknown Source)
     [java]     at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.call(Unknown Source)
     [java]     at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.call(Unknown Source)
     [java]     at java.util.concurrent.Executors$PrivilegedCallable$1.run(Executors.java:493)
     [java]     at java.security.AccessController.doPrivileged(Native Method)
     [java]     at java.util.concurrent.Executors$PrivilegedCallable.call(Executors.java:490)
     [java]     at java.util.concurrent.FutureTask.run(FutureTask.java:262)
     [java]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
     [java]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
     [java]     at java.lang.Thread.run(Thread.java:745)
     [java] </pre>
     [java] <hr /><i><small>Powered by Jetty://</small></i><br/>                                                
     [java] <br/>                                                
     [java] <br/>                        Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer$1 run
     [java] SEVERE: Bad Response Code!: 500
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer$1 run
     [java] INFO: Call back url was: http://192.168.2.26:80/ode2/receivebuild/vb4ftxd7m12fh5urhteyazsvqiieeip5go4p3pnb32w7xna5sbi1zcpggbl3bx8xxtpfuyz4vo1dz7zdjq6s1vwmw2ru69iafami63s2bnueld2wg/build/Android
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer checkMemory
     [java] INFO: Build 1 current used memory: 12697992 bytes
     [java] Feb 24, 2016 11:52:37 AM com.google.appinventor.buildserver.BuildServer$1 run
     [java] INFO: BUILD 1 FINISHED

Thank you, Sajjad

xuwenbin commented 7 years ago

Hello there, I'm trying to compile and run AppInventor 2 on AppScale, but encountered the same error.

It's been several months since this issue was opened, so I'm wondering whether there is progress on providing support to GAE SDK 1.9.x?

Or, at least, hints on how to overcome the compilation error and deploy the bits to AppScale?

llarsson commented 6 years ago

The first line of the project's README says:

AppScale is an open-source cloud computing platform that automatically deploys and scales unmodified Google App Engine applications over public and private cloud systems and on-premise clusters.

Should it be updated to more accurately reflect the state of this issue, or should this currently unassigned, unlabeled, and milestone-less issue be resolved instead?