search

dmwm / WMCore

Core workflow management components for CMS.
Apache License 2.0
46 stars 107 forks source link

fix the returnCursor or remove it #8506

Open ticoann opened 6 years ago

ticoann commented 6 years ago

This causes following error when there more than 600 binding is passed (current cms oracle setup)

DatabaseError: (DatabaseError) ORA-01000: maximum open cursors exceeded.

In the comment, it said it is not supported by select - make sure this is true. https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Database/DBCore.py#L86

There is one place in WMCore it uses but there are few places in DBS uses. @yuyiguo

https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/JobSplitting/JobFactory.py#L289

bbockelm commented 6 years ago

The solution we discussed yesterday was to return an iterator which yields the cursors. That way, the returning code can process them instantly and close them as they are returned.

See also: https://github.com/dmwm/DBS/issues/560

ticoann commented 6 years ago

Thanks Brian, that is good idea. I looked at the code. There are few place needs to be changed to pass the iterable instead of list. However, only when it returns the cursor not data (There are too many place to using this I am not sure whether how many cases it expect list (using list method). Anyway, I will create the patch. Thank you for the tip.