Closed donsizemore closed 4 years ago
@donsizemore thanks for opening this issue. I'd be curious to hear more thoughts from you and others on Payara vs. Glassfish. To me, the big difference is that people installing Dataverse can pay for commercial support from Payara if they wish. Also, Payara seems to release security fixes more often than Glassfish, but I haven't studied this closely.
I wanted to mention that I spoke with @smillidge at JavaOne this past week and he said that because Payara 5 will be a major release (going from 4 to 5) they plan to take this opportunity to incorporate additional features into the product. That is to say, just because Glassfish 5 was released a couple weeks ago doesn't mean that we should expect Payara 5 immediately. (For @smillidge 's information, @donsizemore helps with the care and feeding of https://dataverse.unc.edu and https://dataverse.org contains a map where you can see all 26 installations of Dataverse.)
@donsizemore any logs from failures are useful to the Payara team. The more datapoints we have for people deploying large scale applications like Dataverse to Payara the better.
I'm still thinking about Glassfish vs. Payara, especially as I was poking at #4217 a bit today. I took a look at the release dates at https://en.wikipedia.org/wiki/GlassFish and https://en.wikipedia.org/wiki/Payara_Server and came up with the chart that shows days since previous release:
I think the story it tells (which I already knew to some extent) is that Payara releases much more regularly than Glassfish.
Here's the spreadsheet the chart above is based on: https://docs.google.com/spreadsheets/d/1-BRlfcsByGHmKz8_rkskysSr0LWw8Ou0vBrznCLtb58/edit?usp=sharing
As http://opensourcesurvey.org/2017/ indicates, what users really care about in open source is security and stability:
I suspect that more frequent releases generally means better security. I'm forever patching my Mac and Linux laptops at least.
Also, now that Glassfish is moving to Eclipse, when will the next release be after Glassfish 5.0? https://projects.eclipse.org/projects/ee4j/faq says, "Oracle may deliver maintenance releases of GlassFish 5.0." Glassfish feels a bit in limbo at the moment.
@smillidge any idea when Payara 5 will be out? 😄
Over at pull request #4244 I got Dataverse running on Glassfish 5. There are a couple changes I had to make that I don't full understand ( 6f72baa and 1530273 ) but the app basically works, I think! 😄
@izahn came up with a much better plot for the Glassfish vs. Payara release frequency data I mentioned above (thanks!):
Here's the R script and the data:
I just noticed at https://javabot.evanchooly.com/logs/%23glassfish/2018-03-19 that people are chatting about how Payara 5 is out. They linked to https://twitter.com/Payara_Fish/status/975776806539284480
Heads up that "Payara 5 requires JDK8u161 or above" according to https://github.com/payara/Payara/issues/2513#issuecomment-371914371
Hat tip to @donsizemore for this discovery.
Hi @pdurbin,
back from vacation mentioned in #4248...
Re-Reading the issues about Glassfish and Payara you guys seem to tend to use Payara in the future?
Would this be a good starting point? Or should we just stick with Glassfish 4.1 for now?
Cheers, Oliver
@poikilotherm welcome back! (I just got back from vacation myself.)
I don't believe any Dataverse installations are using Payara yet. I think everyone is using Glassfish 4.1, which came out it 2014. So you should use Glassfish 4.1 for now but if you have an extra server and want to play around with Payara I think it would be interesting to get Dataverse running on an application server like Payara that has a more regular release schedule than Glassfish does.
@poikilotherm thanks for opening #5068 and #5187, for chatting at http://irclog.iq.harvard.edu/dataverse/2018-10-15 , and for agreeing to work on this issue! Please let us know if there's help you need!
Please join discussion at https://groups.google.com/d/msgid/dataverse-dev/0937d60c-b2b7-d74b-1365-16f154d769d8%40fz-juelich.de about favoring Eclipse Glassfish or Payara :smile:
@poikilotherm thanks for that very thoughtful post! I provided my feedback on your post above in two parts:
Any news to share about how the Payara stuff is going? Are you blocked? Do you need anything?
The Payara stuff is going pretty well so far, using it in #5292. I need to get my party started in #5361 but of course one could try to use Payara 5 using classic installation mode (altough the installer needs to be adapted in a few places, because the production
domain is favored for prod use)
Does Payara 5 support Java 11? I'm confused about the situation and wrote a bit about it at https://github.com/IQSS/dataverse/issues/5512#issuecomment-462957172
I just learned that Payara has an (unmerged) branch for Java 11 support, which I just wrote about at https://github.com/payara/Payara/issues/2296#issuecomment-478334106
I just ran...
ec2-create-instance.sh -g main.yml -r https://github.com/smillidge/dataverse.git -b 5893-make-flyway-ejb-valid
... that branch is by @smillidge at https://github.com/IQSS/dataverse/pull/5894
... with my slightly hacked version of that script: ec2-create-instance.sh.txt
The diff shows the hacks below and they are for these issues:
Teams-iMac:bin family$ diff -u ec2-create-instance.sh.2019-05-30 ec2-create-instance.sh
--- ec2-create-instance.sh.2019-05-30 2019-05-30 18:17:35.000000000 -0400
+++ ec2-create-instance.sh 2019-05-31 09:04:57.000000000 -0400
@@ -130,8 +130,12 @@
# TODO: Add some error checking for this ssh command.
ssh -T -i $PEM_FILE -o 'StrictHostKeyChecking no' -o 'UserKnownHostsFile=/dev/null' -o 'ConnectTimeout=300' $USER_AT_HOST <<EOF
sudo yum -y install epel-release
-sudo yum -y install git nano ansible
+sudo yum -y install https://releases.ansible.com/ansible/rpm/release/epel-7-x86_64/ansible-2.7.9-1.el7.ans.noarch.rpm
+sudo yum -y install git nano
git clone https://github.com/IQSS/dataverse-ansible.git dataverse
+cd dataverse
+git checkout 43-payara5
+cd ..
export ANSIBLE_ROLES_PATH=.
echo $extra_vars
ansible-playbook -v -i dataverse/inventory dataverse/dataverse.pb --connection=local $GVARG
Teams-iMac:bin family$
Here's my main.yml (renamed so I can upload it here): main.yml.txt
Deployment worked...
Branch "5893-make-flyway-ejb-valid" from https://github.com/smillidge/dataverse.git has been deployed to http://ec2-34-207-192-222.compute-1.amazonaws.com:8080 When you are done, please terminate your instance with: aws ec2 terminate-instances --instance-ids i-0fcfa00235f085e06
... but I'm seeing errors on the home page:
Here are the server.log files:
Lots of errors in server.log to dig into.
Then I logged in as dataverseAdmin and went to http://ec2-34-207-192-222.compute-1.amazonaws.com:8080/dataverse/root where I got a 500 error:
The bottom line for me is that thanks to @donsizemore we have a relatively easy way to spin up Dataverse on arbitrary versions of Glassfish or Payara thanks to the work he has put into https://github.com/IQSS/dataverse-ansible/issues/43 . Of course we are also building on his previous work of allowing us to spin up arbitrary branches in https://github.com/IQSS/dataverse-ansible/issues/22 but that's somewhat old news at this point. 😄
This issue is not in our sprint over at https://github.com/orgs/IQSS/projects/2#column-5298408 but since we have the attention of @smillidge (the founder of @Payara), maybe it should be. Any thoughts on this @djbrooke (who might be off on a week of vacation already, since he's in the future). 😄
OK I got the same error. I assumed as it was coming from a SearchServiceBean
and I hadn't installed Solr it was related to that. As a Payara engineer the error is coming from a business logic bean SearchServiceBean
so I don't know whether it is a server problem or application problem.
Speaking of search, this ancient unmerged commit but might be related:
page
to "0" on Glassfish 5. Why? #4217Also, for the weird validation error on the home page, here's another ancient unmerged commit:
A breakpoint on setPage would tell you who or what sets it to 0. (teaching you to suck eggs)
I investigated this. Page is being set to zero by this line in dataverse.xhtml
<f:viewParam name="page" value="#{SearchIncludeFragment.page}"/>
essentially there is not GET value of page
so it is null and being coerced to zero. This may just be due to an update in the EL 3.0.
The weird validation error is again because alias is null in the line
<f:viewParam name="alias" value="#{DataversePage.dataverse.alias}"/>
again not sure why that is causing validation errors as it isn't marked as required.
It's being coerced to zero because of
<context-param>
<param-name>
javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL
</param-name>
<param-value>true</param-value>
</context-param>
Heh. Nice shirt, @poikilotherm https://twitter.com/poi_ki_lo_therm/status/1135506939465654272
@poikilotherm is a community contributor and the only one assigned to this issue. @donsizemore said he's an interested participant over at http://irclog.iq.harvard.edu/dataverse/2019-05-30#i_95419 😄
If there are others would would like to help with this effort of getting Dataverse running on Payara, please coordinate with @poikilotherm . I'm happy to add you to a "team" here on GitHub and co-assign you to this issue! I'd love to get this moving from "Community Dev" to "Code Review" over at our kanban board at https://github.com/orgs/IQSS/projects/2 . At standup on Friday I suggested pulling this issue into the IQSS sprint to strike while the iron is hot but we have other stuff we're working on right now. That said, I and others can certainly help support the community effort! 🚀
@smillidge thanks for all the work you've put into this, especially over the weekend! 🎉
I opened IQSS/dataverse-kubernetes#63 and IQSS/dataverse-kubernetes#64 to keep track of adding support for this also in K8s right away... :wink:
(And yes, @pdurbin, that shirt was by intent :smile: )
The weird validation error is again because alias is null in the line
<f:viewParam name="alias" value="#{DataversePage.dataverse.alias}"/>
again not sure why that is causing validation errors as it isn't marked as required.
@smillidge I'm pretty sure the alias validation error in the screenshot below is coming from @NotBlank(message = "{dataverse.alias}")
here:
I say this because when I comment it out the error below goes away:
However, we have that bean validation annotation in there for a reason. When a user creates a dataverse, we want an alias to be entered. Here's the validation error we show for alias under Glassfish 4.1 when the user doesn't fill in an alias (from the demo site):
So, we can't simply comment out @NotBlank(message = "{dataverse.alias}")
. We need it or something like it to work on Payara 5.
Problem is the GET request does not have an alias present in the request so it is being set to null and I think propagated through somewhere to the validation on that first screen. I think is because of the
javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL
setting in the web.xml.
I'm no JSF expert though.
setting javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=false
fixes it and makes the validation error go away. However as this is application wide I am not sure if any other code is relying on it.
@smillidge me neither but please don't let that stop you from making a pull request! :smile:
setting
javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=false
fixes it
I just made pull request #5908 for this. Small chunks. :smile:
Actually I can fix that error using omnifaces
<o:viewParam name="alias" value="#{DataversePage.dataverse.alias}" default="root"/>
@smillidge that's a good thought and in pull request #5913 I just tried setting default to root (I had to upgrade to a newer version of Omnifaces) but as I've described in that pull request the "Create Dataverse" button doesn't work. I don't know if this is because I'm setting the default to root or not, unfortunately.
could well be. I don't know enough about the functional application as I'm approaching this purely as a server engineer :-)
Actually I can fix that error using omnifaces
<o:viewParam name="alias" value="#{DataversePage.dataverse.alias}" default="root"/>
@smillidge just an update that we tried this (merged it even) but discovered that clicking "Add" then "New Dataverse" stopped working. So we had to revert this fix yesterday. That was in pull request #5944.
setting
javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=false
fixes it and makes the validation error go away.
I tried this in pull request #5908 and I agree with you that the validation error goes away. In addition, the pagination error goes away. That's when I stopped testing and sent it to code review. I just suggested at https://github.com/IQSS/dataverse/pull/5908#issuecomment-502067715 that perhaps we should revisit that fix because a long article at http://balusc.omnifaces.org/2015/10/the-empty-string-madness.html has me questioning how well INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=true
works on Glassfish 4.1 anyway.
I definitely feel like I'm still grasping at straws and I don't understand what changed between Glassfish 4.1 and newer versions. It's frustrating to feel stuck on a specific version of a single application server.
I think it is the empty GET param handling in the application. It probably needs to be handled in the business logic layer. For example for the page being zero the service was throwing an IllegalArgumentException
which is pretty brutal. Defensive programming could be added to just set any value of page <= 0 to 1 in the service rather than throwing an error to the higher layers.
My guess is that for the alias parts of the application are expecting root but other parts e.g. "New Dataverse" are expecting the name to be null or empty string.
If you are stuck on a very specific version of GlassFish I suspect you are relying on the EL bug referred to in Balus' blog.
Defensive programming could be added to just set any value of page <= 0 to 1 in the service rather than throwing an error to the higher layers.
Yes, and yesterday I made pull request #5940 for exactly this.
My guess is that for the alias parts of the application are expecting root but other parts e.g. "New Dataverse" are expecting the name to be null or empty string.
Yes, probably. We need a better pull request than #5933 but I'm not sure what the "diff" will look like yet.
If you are stuck on a very specific version of GlassFish I suspect you are relying on the EL bug referred to in Balus' blog.
I believe this is sad but true. 😢
@smillidge thanks as always for your insight!!
I just stumbled over https://bugs.eclipse.org/bugs/show_bug.cgi?id=429992 in b0d9c66 . We need to get off old smelling fishes...
Closing in favor of #6230
Umbrella issue to track progress in deploying Dataverse under Payara 5, which is in Alpha as of this writing. The installation currently breaks down in deploying the warfile when creating/starting the EJB timers. I can post/send logs, but Payara 5 is still in Alpha.