Data are not persisted into cassandra

[hsn10]

Calling em.flush() do not saves data into cassandra. Only metadata CF are created. I tested my entity model and sample code in standalone application and it works as expected. Need to debug playorm a bit to see why its ignored.

[deanhiller]

odd, we have not run into em.flush ever not working as of yet. Is this the in-memory version or directly to cassandra? When you checkout playorm, there is a test suite that runs against in-memory to start with but if you go into FactorySingleton.java and lines 23-25 uncomment cassandra and comment out in-memory then all tests will run against cassandra instead. There is 107 tests and that number is always increasing. Let us know if they fail for you and you can compare with your code as well.

[hsn10]

only test failing if i run gradle test is testBasicLocalTime.

junit.framework.AssertionFailedError: expected:<0> but was:<1> at junit.framework.Assert.fail(Assert.java:50) at junit.framework.Assert.failNotEquals(Assert.java:287) at junit.framework.Assert.assertEquals(Assert.java:67) at junit.framework.Assert.assertEquals(Assert.java:199) at junit.framework.Assert.assertEquals(Assert.java:205) at com.alvazan.test.TestIndexTypes.testBasicLocalTime(TestIndexTypes.java:76)

but these tests are not using cassandra, how to point these tests to my cassandra server?

[hsn10]

error was: expected 0 but was 1

[hsn10]

that one error is with inmemory db. i re-read your reply and testing cassandra now.

[hsn10]

with cassandra i have zero test failures.

[deanhiller]

hmmmm, so if you port some of your code into our test suite, is it working or failing?

or vice versa, if you port some of our code into your test suite. Does your code at least work with in-memory version or is it failing on both. Are you 100% sure flush was not in an if statement or something that was not called....just trying to think of anything that could cause that.

[hsn10]

It was usage error. I was under impression that it works like in Hibernate where session is binded to calling thread:

In hibernate following style works:

one function called from controller
NoSql.em().put(token);

in other function:
NoSql.em().flush()

while in your playorm em() always returns new instance

[hsn10]

so only result of this thread is catching one failing unit test. I will report it to playorm and close this.

[deanhiller]

sounds like a bug, NoSql.em() is for playframework only and in 1.2.x it results in the same EntityManager IF the thread is the same so this sounds like purely a 2.x plugin bug.

Dean

[deanhiller]

Vikas, can you look into this as I have never touched the 2.x plugin as of yet.

[hsn10]

where is code for 1.x plugin? i will check it

[deanhiller]

it is in com.alvazan.play package in the playorm project itself.

[hsn10]

You are right. Play1 plugin is using thread local storage. Its simple to code, i just need to check play2 api for providing hooks for cleaning local context.

[hsn10]

Play2 plugin API does not have support for beforeInvocation, afterInvocation and similar.

[hsn10]

probably filter has to be written - http://www.playframework.com/documentation/2.1.2-RC2/ScalaHttpFilters Only problem is that for filter activation its needed to extend Global object and user already can have own Global. Its unclear if application can have more then one GlobalSettings class

[hsn10]

Okay i have code for this. It needs to be more tested in multithreading tasks if race conditions are not here.

[hsn10]

it has race conditions. In heavy multithreaded workload about 10% of updates are lost because it binds new entity manager to thread, overwriting old one. Play framework 2 is using async replies which makes it difficult to init entity manager because it will be called later by other thread.

[deanhiller]

well, play 1.2.x had the same one request can call controller multiple times so we stuck with the simple one NoSqlEntityManager per controller method call instead which seems to work out fine as generally you flush at the end of the controller method call. Is there anything like beforeInvocation and afterInvocation possibly in play 2.x?

[hsn10]

currently best known fix for me is to do that: It can run my tests without data loss.

public NoSqlEntityManager em() {
    NoSqlEntityManager em = entityManager.get();
    if(em != null)
        return em;

    if(entityManagerFactory == null) {
        throw new RuntimeException("No NoSqlEntityManagerFactory configured");
    }
    System.out.println("creating and binding NEW em");
    em = entityManagerFactory.createEntityManager();
    entityManager.set(em);
    return em;
}

[hsn10]

second part of testing is making sure that not flushed data are discarded. If not, then there is need to add api for discarding data like em.rollback()

[deanhiller]

Vikas, can you post on the playframework 2.x list if there is a way to clear the ThreadLocal after a controller has been called (sort of like a afterInvocation method) such that we can clear the entitymanager for that thread.

[hsn10]

it looks like this: cleanup action is not guaranteed to be called from same thread as controller action.

Thread[play-akka.actor.default-dispatcher-6,5,main] saving 9019189733991783391 Thread[play-akka.actor.default-dispatcher-8,5,main] saving -5357047405576633570 Thread[play-akka.actor.default-dispatcher-7,5,main] saving 3636428833826798739 Thread[play-akka.actor.default-dispatcher-4,5,main] saving 4488436642740285347 Thread[play-akka.actor.default-dispatcher-2,5,main] saving 5356246889603830323 Thread[play-akka.actor.default-dispatcher-9,5,main] saving -7824999486368613868 Thread[play-akka.actor.default-dispatcher-8,5,main] saving 2908257382006788523 Thread[play-akka.actor.default-dispatcher-3,5,main] saving -494104006919642858 Thread[play-akka.actor.default-dispatcher-5,5,main] saving 6275055572434286109 Thread[play-akka.actor.default-dispatcher-5,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-5,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-5,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-9,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-8,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-4,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-2,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-3,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-2,5,main] unbinding async result Thread[play-akka.actor.default-dispatcher-6,5,main] saving 3642727928594026527 Thread[play-akka.actor.default-dispatcher-6,5,main] unbinding async result

[hsn10]

as i see it, best would be to make NoSql.em() always create new object and user has to pass it as argument. I tried both http://www.playframework.com/documentation/2.1.2-RC2/ScalaActionsComposition and http://www.playframework.com/documentation/2.1.2-RC2/ScalaHttpFilters and none guarantee that hooks will be called by same thread. Play framework 2.0 is based on actor async model.

[deanhiller]

okay, another idea. What we really want to do is tie the EntityManager to the request lifecycle, correct? such taht request created, entitymanager created if needed and destroyed when request is destroyed. Do they have something where you could in NoSql.em() call Request.currentRequest.setParam("em", em) and when the request goes away the EntityManager naturally gets garbage collected for you....nothing for you to do at all.

[hsn10]

This is good idea. I am checking Scala API and Request http://www.playframework.com/documentation/api/2.1.1/scala/index.html#play.api.mvc.Request can have just map [String, String] attached, it can not have arbitrary object. Possibility is to use Request.Id:Long and create hashtable(id,em) for storing entitymanager based on request id. I need to check stability of that id field, if adding or removing headers keep it same.

[deanhiller]

hmmmm, in that case, you would need to know when the request is finished so you can erase the usage from the hashtable :(. In play 1.2.x, they had a Map<String, Object> that we store stuff in....I wonder why they still don't have that so that you don't have to worry about cleaning up the hashtable and leaking memory there.

[hsn10]

Cleaning works fine, you can see it from my dump. They still have map but its [String, String]

[hsn10]

map is Request.tags[String, String]

[hsn10]

request.id is stable during request lifecycle. it can be used for that purpose. But it needs to be passed to play.NoSql.em() somehow. What method do you propose

[deanhiller]

needs to be passed to NoSql.em()?

Is there no Request.current() like in the previous playframework?

Also, out of curiosity, is there a way to use the "pimp your library (scala pattern)" of scala to add a field in the Request object of Map<String, Object> so you don't need a static Map to clean up?

How are you going to clean up the static Map<String, EntityMgr> idToMgr so it doesn't keep growing?

[hsn10]

I am thinking about this http://underflow.ca/blog/798/playing-with-actions/ wrap action and add EntityManager parameter. It will not be invisible solution like filter, but it should fit playframework programming style nicely.

[deanhiller]

If you need to change the api of NoSql, let's just create a NoSql2 and so we would accept any pull requests for a NoSql2 that you could use NoSql -> play 1.2.x and NoSql2 -> play 2.x. After all, I think you have more knowledge of the play 2.x line than I do. I am a bit rusty on scala as well though I do prefer scala to java as I like the immutability and the conciseness of alot of stuff.

later, Dean

[hsn10]

since i could not find any magic how to do it transparently, i just wrapped action. Because scala and java have different api there have to be 2 plugins: main for scala with this and small one (just 1 class) for java.

import play.api.mvc._ import com.alvazan.orm.api.base.NoSqlEntityManager

object ActionHelpers {

def withEntityManager(action: NoSqlEntityManager => EssentialAction):EssentialAction = { EssentialAction { request => action(NoSqlForPlay2.getEntityManagerFactory().createEntityManager())(request) } } }

[hsn10]

its simple to use in controller, just wrap action withEntityManager. No need to cleanup, gc will care.

def index = withEntityManager { mgr => Action { implicit request => val token = TokenTools.generate() mgr.put(token) mgr.flush() println(Thread.currentThread()+ "/"+request.id+ " saving "+token) Ok(views.html.index( postForm.fill( (new Note(), token)) )) } }

[deanhiller]

sounds fine to me. Just do a pull request or I guess you can wait until Vikas wakes up. I would not even know where to put the code and don't really have 5 minutes to look though if you do a pull request, I will gladly click the button that merges it ;).

[hsn10]

ok, so create another project play2plugin-java

[easility]

I can create another project but can we do: NoSql.em() for Java for both Play1.x and Play2.x like existing and NoSql2.em() for Scala for Play2.x? In that case, will we need another project?

[hsn10]

I would leave play 1.x plugin where it is now. It works fine in play 1 / java and i never used play1 with scala.

I will read documentation about sbt and make multi project sbt build in play2plugin. It will produce 2 jars, one for java second for scala

[deanhiller]

sounds good to me.

[hsn10]

I did variant with Global object, but i will probably change it to this http://www.playframework.com/documentation/2.1.1/JavaActionsComposition - it will be consisent with scala version and allows user to define own global object.

[deanhiller]

should I close this now or?