search

marklogic-community / marklogic-samplestack

A sample implementation of the MarkLogic Reference Architecture
Apache License 2.0
82 stars 56 forks source link

Middle tier leaves open transactions. #218

Closed grechaw closed 9 years ago

grechaw commented 9 years ago

After running unit tests, the appserver has open transactions it shouldn't.

This prevents teardown, among other probably issues.

grechaw commented 9 years ago

I have located this transaction leak within the server-side javascript response transform. Not sure yet whether its a REST API bug or a bug in my transform implementation.

grechaw commented 9 years ago

Created bug 29915 in bugtrack as this is not as samplestack bug. I'm closing this as duplicate.

laurelnaiad commented 9 years ago

I have bugs that will go live when/if I merge until this is fixed. Regardless of reason, our process isn't working properly, so let's leave this open for traceability.

popzip commented 9 years ago

agreed i'd like to keep the dependency here.

grechaw commented 9 years ago

What does "external" do to bugtrack?

laurelnaiad commented 9 years ago

AFAIK there's such a status in BT and so the BT bug should reflect it.

grechaw commented 9 years ago

"External" works for me -- that way you can see the relationship between the two bugtrack tickets and the status reporting doesnt have a hidden duplicate.

So I haven't been able to isolate this bug in REST or JavaScript yet, but if I simply use an XQuery transform instead of JavaScript the extra transactions are not created. I have queries in to Jane. The transactions that are kept around are such that they've probably gone unnoticed until now - they are read transactions and don't interfere with other things on the server, unless you try to drop a database.

QA hasn't gotten to server-side javascript extensions for REST yet, so I think once again I (actually, @stu-salsbury ) was the first to find this bug.

Depending on further triage today, this bug fix may or may not be available for EA-3. We should consider that as a possibility; I'm going to see what Jane has to contribute, and in the meantime try to work on implementing and testing /v1/tags

laurelnaiad commented 9 years ago

If the server is leaving dangling transactions when we use Javascript extensions, then I really hope we can fix it before EA3 is released in November. That functionality has a long tail as demonstrated here.

popzip commented 9 years ago

Maybe a good topic for Tuesday - if we're leading the server testing then it might be jumping the gun to hope for it to be ready in EA3. Let's keep close watch on it and maybe we can get this bug fix prioritized. Otherwise the default is not to have the open transactions, but to use XQuery instead.

grechaw commented 9 years ago

I'm getting my work in order right now to have a good perspective on my bug and task list,

but as far as this one goes, if the bug turns out to be in the REST layer, I'll be on the hook to fix it and it pushes on tags and related tags.

If it turns out to be in the server layer then it impacts Jane, who is already up to her ears in EA-3 tasks.

I'll know more this afternoon. Ideally it will be my bug and very simple to fix.

From: Kasey Alderete [notifications@github.com] Sent: Friday, October 17, 2014 11:04 AM To: marklogic/marklogic-samplestack Cc: Charles Greer Subject: Re: [marklogic-samplestack] Middle tier leaves open transactions. (#218)

Maybe a good topic for Tuesday - if we're leading the server testing then it might be jumping the gun to hope for it to be ready in EA3. Let's keep close watch on it and maybe we can get this bug fix prioritized. Otherwise the default is not to have the open transactions, but to use XQuery instead.

— Reply to this email directly or view it on GitHubhttps://github.com/marklogic/marklogic-samplestack/issues/218#issuecomment-59551880.

laurelnaiad commented 9 years ago

Ideally it will be my bug

A true glutton for punishment!

grechaw commented 9 years ago

Erik H had a vague memory of a similar bug inXQuery uncovered last year. He, Jane and I will convene to get mutual understanding.

grechaw commented 9 years ago

Reports are that the blocking bug in JavaScript is fixed. I'll be able to verify, and if so, to move to using JavaScript extensions in the next PR.

popzip commented 9 years ago

Is this still a possible blocker?

grechaw commented 9 years ago

I haven't been able to verify, but this is presumed fixed on the server. To verify I have to move back to using JavaScript transforms. I expect this to happen by EA-3, but not this week.

laurelnaiad commented 9 years ago

I don't know if this is related, but as of today's nightly, I still can't succeed with dbInit after dbteardown.

:buildSrc:compileJava UP-TO-DATE
:buildSrc:compileGroovy UP-TO-DATE
:buildSrc:processResources UP-TO-DATE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar UP-TO-DATE
:buildSrc:assemble UP-TO-DATE
:buildSrc:compileTestJava UP-TO-DATE
:buildSrc:compileTestGroovy UP-TO-DATE
:buildSrc:processTestResources UP-TO-DATE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test UP-TO-DATE
:buildSrc:check UP-TO-DATE
:buildSrc:build UP-TO-DATE
:dbTeardown
Waiting for server restart...

BUILD SUCCESSFUL

Total time: 12.188 secs
:buildSrc:compileJava UP-TO-DATE
:buildSrc:compileGroovy UP-TO-DATE
:buildSrc:processResources UP-TO-DATE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar UP-TO-DATE
:buildSrc:assemble UP-TO-DATE
:buildSrc:compileTestJava UP-TO-DATE
:buildSrc:compileTestGroovy UP-TO-DATE
:buildSrc:processTestResources UP-TO-DATE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test UP-TO-DATE
:buildSrc:check UP-TO-DATE
:buildSrc:build UP-TO-DATE
:clean

BUILD SUCCESSFUL

Total time: 2.327 secs
:buildSrc:compileJava UP-TO-DATE
:buildSrc:compileGroovy UP-TO-DATE
:buildSrc:processResources UP-TO-DATE
:buildSrc:classes UP-TO-DATE
:buildSrc:jar UP-TO-DATE
:buildSrc:assemble UP-TO-DATE
:buildSrc:compileTestJava UP-TO-DATE
:buildSrc:compileTestGroovy UP-TO-DATE
:buildSrc:processTestResources UP-TO-DATE
:buildSrc:testClasses UP-TO-DATE
:buildSrc:test UP-TO-DATE
:buildSrc:check UP-TO-DATE
:buildSrc:build UP-TO-DATE
:dbConfigureClean
:dbInit
Error parsing 'text/html; charset=utf-8' response
groovy.json.JsonException: Unable to determine the current character, it is not a string, number, array, or object

The current character read is '<' with an int value of 60
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
<html xmlns="http://www.w3.org/1999/xhtml">
^
    at groovy.json.internal.JsonParserCharArray.decodeValueInternal(JsonParserCharArray.java:216)
    at groovy.json.internal.JsonParserCharArray.decodeValue(JsonParserCharArray.java:166)
    at groovy.json.internal.JsonParserCharArray.decodeFromChars(JsonParserCharArray.java:45)
    at groovy.json.internal.JsonParserCharArray.parse(JsonParserCharArray.java:409)
    at groovy.json.internal.BaseJsonParser.parse(BaseJsonParser.java:121)
    at groovy.json.JsonSlurper.parse(JsonSlurper.java:224)
    at groovyx.net.http.ParserRegistry.parseJSON(ParserRegistry.java:280)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:324)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1207)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1074)
    at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1016)
    at groovy.lang.Closure.call(Closure.java:423)
    at groovy.lang.Closure.call(Closure.java:439)
    at groovyx.net.http.HTTPBuilder.parseResponse(HTTPBuilder.java:551)
    at groovyx.net.http.HTTPBuilder$1.handleResponse(HTTPBuilder.java:480)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1070)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1044)
    at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:506)
    at groovyx.net.http.RESTClient.post(RESTClient.java:141)
    at groovyx.net.http.RESTClient$post.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
    at MarkLogicInitTask.adminInit(MarkLogicInitTask.groovy:28)
    at MarkLogicInitTask.initMarkLogic(MarkLogicInitTask.groovy:16)
laurelnaiad commented 9 years ago

Obviously this at least looks like the HTML content-type changes that Chris put in didn't fully address the issue or they didn't make it into last nightly. But given that it's presumably an error message, there seems to be a root cause in addition to the content-type exacerbation.

grechaw commented 9 years ago

Yeah the commits were from just a few minutes ago, so look for it tomorrow

laurelnaiad commented 9 years ago

So is there no underlying issue here? It seems to be failing to post to ":8001/admin/v1/init" successfuly, which would suggest that there's a bigger problem.

laurelnaiad commented 9 years ago

Or is that the call that is supposed to fail if the server has already been initialized.... I guess that's it, huh? So nevermind...

jmakeig commented 9 years ago

@ndw recently changed the error output for this exact scenario. Is Gradle expecting the old response?

laurelnaiad commented 9 years ago

Justin I will forward an email exchange on the subject for your review.

grechaw commented 9 years ago

What's the HTML stuff doing in a bug about open transactions? This is the bug that was preventing teardown. Not important never mind

laurelnaiad commented 9 years ago

It's here because it crops up onlly when I try to get away with not wiping out my data directory, and I associated it with dbTeardown which takes the place of wiping out one's data directory. Essentially, I mentioned it prior to realizing that it's happening because the server is trying to say that it's already been initialized.

laurelnaiad commented 9 years ago

Whether or not dbTeardown is leaving open transactions is a bit moot now, since it includes a MarkLogic restart. @grechaw it seems like we might be able to move this to test?

grechaw commented 9 years ago

Well, root cause for this issue originally was the use of JavaScript extensions. I've still not verified that using JavaScript extensions is safe wrt leaving transactions open. The number of transactions in that case was one-per-request which would cause serious problems. When I switch to JS extensions again, we'll either see it fixed or not; I expect it has been fixed. I plan to do that today at some point --

Once I've verified that JS extensions don't leak transactions I'll put this in test with info for Gaurav about how to test.

Sound good?

grechaw commented 9 years ago

I have verified that we can use JavaScript transforms without this transaction leak.

jmakeig commented 9 years ago

Yeah!