search

TEAMMATES / teammates

This is the project website for the TEAMMATES feedback management tool for education
https://teammatesv4.appspot.com/
GNU General Public License v2.0
1.65k stars 3.28k forks source link

Error message in live server: Not valid UTF8 #10702

Closed damithc closed 3 years ago

damithc commented 4 years ago

Error message from live server image

Note: in all cases of this nature, the user agent seems to be WhatsApp

org.eclipse.jetty.util.Utf8Appendable$NotUtf8Exception: Not valid UTF8! char appended in state 84 at org.eclipse.jetty.util.Utf8Appendable.checkCharAppend(Utf8Appendable.java:109) at org.eclipse.jetty.util.Utf8Appendable.append(Utf8Appendable.java:117) at org.eclipse.jetty.util.UrlEncoded.decodeUtf8To(UrlEncoded.java:363) at org.eclipse.jetty.util.UrlEncoded.decodeTo(UrlEncoded.java:226) at org.eclipse.jetty.util.UrlEncoded.decodeTo(UrlEncoded.java:210) at org.eclipse.jetty.server.Request.mergeQueryParameters(Request.java:2407) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:195) at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:74) at teammates.ui.webapi.action.WebPageServlet.doGet(WebPageServlet.java:38) at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772) at com.googlecode.objectify.ObjectifyFilter.doFilter(ObjectifyFilter.java:48) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at com.google.apphosting.runtime.jetty9.ParseBlobUploadHandler.handle(ParseBlobUploadHandler.java:119) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1182) at com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.doHandle(AppEngineWebAppContext.java:187) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:293) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:539) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) at com.google.apphosting.runtime.jetty9.RpcConnection.handle(RpcConnection.java:213) at com.google.apphosting.runtime.jetty9.RpcConnector.serviceRequest(RpcConnector.java:81) at com.google.apphosting.runtime.jetty9.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134) at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchServletRequest(JavaRuntime.java:761) at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchRequest(JavaRuntime.java:724) at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:694) at com.google.apphosting.runtime.JavaRuntime$NullSandboxRequestRunnable.run(JavaRuntime.java:880) at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:270) at java.lang.Thread.run(Thread.java:748)

wkurniawan07 commented 4 years ago

What even is a WhatsApp user agent??

damithc commented 4 years ago

What even is a WhatsApp user agent??

No idea. Maybe some WhatsApp apps can open a web page inside it?

Wimpzer commented 3 years ago

The first post I find on Google in regards to WhatsApp User-agent is that it is a bot doing HTTP requests. However, there are different answers depending on the number after WhatsApp/2.??.??.

In regards to the error, I would imagine the invalid UFT8 char is coming from the user agent and therefore not an error with the code. However, a possible fix would be to enforce another encoding which allows for the characters which are throwing the error.

wkurniawan07 commented 3 years ago

I managed to reproduce the issue by including invalid UTF-8 character in one of the request parameters. However, no sane modern browser that respects the UTF-8 encoding will produce this kind of error.

For reference, the same situation in V6 will look like this:

Screen Shot 2020-10-05 at 12 13 47 AM

It remains an issue that the exception is thrown as 500 instead of 400, and it looks like it's thrown from deep within the Jetty library. The best we can do is catch that exception (if we can) and change the status code to 400 to indicate that this is a user error.

ChooJeremy commented 3 years ago

I managed to reproduce the issue by including invalid UTF-8 character in one of the request parameters. However, no sane modern browser that respects the UTF-8 encoding will produce this kind of error.

For reference, the same situation in V6 will look like this:

Screen Shot 2020-10-05 at 12 13 47 AM

It remains an issue that the exception is thrown as 500 instead of 400, and it looks like it's thrown from deep within the Jetty library. The best we can do is catch that exception (if we can) and change the status code to 400 to indicate that this is a user error.

How are you testing this in v7? I'm trying the following: http://localhost:4200/web/instructor/sessions/edit?courseid=test.ema-demo&fsname=Second%20team%20fe%F3dback%20session It redirects me to the home page. If I test it on the server side (localhost:8080) it correctly returns 400, not 500.

wkurniawan07 commented 3 years ago

http://localhost:4200/web/instructor/sessions/edit?courseid=test.ema-demo&fsname=Second%20team%20fe%F3dback%20session

This is expected as Angular dev server knows nothing of our back-end server.

If I test it on the server side (localhost:8080) it correctly returns 400, not 500

Local Jetty server works differently from production server (another reason why GAE Java 8 is not very good). To test this change unfortunately we have no choice but to deploy and observe. The URL you showed (after changing the host) is a valid test candidate. You might want to test calling some web API endpoints as well.