search

COOL-cohort / COOL

the source code of the COOL system
https://www.comp.nus.edu.sg/~dbsystem/cool/
Apache License 2.0
45 stars 16 forks source link

Add distributed server, add comments, and clean code #34

Closed NLGithubWP closed 2 years ago

NLGithubWP commented 2 years ago
  1. Add distributed server with broker, worker, zookeeper, and hdfs.
  2. Replace server (jetty) with SpringBoot
  3. Add comments
  4. Clean code
  5. Remove dimension variable
  6. document is at https://github.com/COOL-cohort/COOL-site/pull/22
NLGithubWP commented 2 years ago

I reviewed some code about io and storage layout. Do not delete test data file under test/resources/TestData @NLGithubWP

OK, Added

Zrealshadow commented 2 years ago

I reviewed some code about io and storage layout. Do not delete test data file under test/resources/TestData @NLGithubWP

OK, Added

This PR works in my side. For changes in part of io and storage layout, it is acceptable. If you want to merge, do not forget to rebase or squash your commit log, which can help us manage the history of commit.

NLGithubWP commented 2 years ago

I reviewed some code about io and storage layout. Do not delete test data file under test/resources/TestData @NLGithubWP

OK, Added

This PR works in my side. For changes in part of io and storage layout, it is acceptable. If you want to merge, do not forget to rebase or squash your commit log, which can help us manage the history of commit.

Rebased

KimballCai commented 2 years ago

The server has been changed from jetty to Springboot

KimballCai commented 2 years ago

This PR also removes the need for the dimension file.

hugy718 commented 2 years ago

Just tried the build the project and test the standalone spring server and found the following issues

KimballCai commented 2 years ago

Just tried the build the project and test the standalone spring server and found the following issues

  • removal of the shade plugin broke our command line samples. (dependencies not packaged into the jar). However if I add back the shade plugin, I am not able to launch spring server.
022-04-28 20:38:13.592 ERROR 16618 --- [           main] o.s.b.w.e.t.TomcatStarter                : Error starting Tomcat context. Exception: org.springframework.beans.factory.BeanCreationException. Message: Error creating bean with name 'formContentFilter' defined in class path resource [org/springframework/boot/autoconfigure/web/servlet/WebMvcAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.web.servlet.filter.OrderedFormContentFilter]: Factory method 'formContentFilter' threw exception; nested exception is java.lang.NoClassDefFoundError: Could not initialize class com.fasterxml.jackson.annotation.JsonInclude$Value
  • Incorrect result for health data set selection. I added back the shade plugin and tested the different variation of loading, sogamo query0 and health query1-0. The result of query 1-0 differs from current head of main. (p-1 missing). We need to make sure the results are consistent. @KimballCai @NLGithubWP please check the aggregation, funnel, iceberg results.
  • Standalone server API: listing and loading seems working. But i cannot run the selection and analysis APIs.

I have checked this problem. It is correct that P-1 is missing because we have defined the time window where selected patients must have records of more than 7 days. This problem has been discussed in PR #25.

{"timestamp":"2022-04-28T13:06:12.417+00:00","status":415,"error":"Unsupported Media Type","path":"/cohort/selection"}

or

Failed to open/read local data from file/application

even the file exist under the directory

hugy718 commented 2 years ago

Nice