dee1024 / housedb

🏡🏡🏡 房源数据爬虫。支持断点续爬,价格变更通知,提供数据的分析统计服务。
318 stars 84 forks source link

如何部署 #1

Open matrixxun opened 6 years ago

matrixxun commented 6 years ago

能简单写一下如何部署使用的教程么?

dee1024 commented 6 years ago

简单描述下,希望可以明白。 1.按照springboot的规范部署; 2.启动后可以开始自动抓取数据并且写房源信息到log文件; 3.启动ELK载入实时分析房源log文件,各种聚合分析出报表。

hx462 commented 6 years ago

运行了半个小时,一直在启动和关闭HikariPool,没有抓取任何数据。测试用例运行正常,可以抓到数据。我查了“sqlSessionFactory”这个报错,说是我代码中有重名的问题,可是具体要怎么排除错误不太清楚。。 还望指点一下!

2018-04-08 11:09:03.918 INFO 6081 --- [restartedMain] c.g.c.sloth.lianjiadb.Application : Starting Application on ShawndeMacBook-Pro.local with PID 6081 (/Users/shawn/IdeaProjects/housedb/target/classes started by shawn in /Users/shawn/IdeaProjects/housedb) 2018-04-08 11:09:03.924 INFO 6081 --- [restartedMain] c.g.c.sloth.lianjiadb.Application : No active profile set, falling back to default profiles: default 2018-04-08 11:09:03.987 INFO 6081 --- [restartedMain] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@2d304b97: startup date [Sun Apr 08 11:09:03 CST 2018]; root of context hierarchy 2018-04-08 11:09:04.238 INFO 6081 --- [background-preinit] o.h.validator.internal.util.Version : HV000001: Hibernate Validator 5.2.2.Final 2018-04-08 11:09:05.081 INFO 6081 --- [restartedMain] o.s.b.f.s.DefaultListableBeanFactory : Overriding bean definition for bean 'beanNameViewResolver' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter.class]] 2018-04-08 11:09:05.593 INFO 6081 --- [restartedMain] trationDelegate$BeanPostProcessorChecker : Bean 'DBConfig' of type [class com.github.coolcool.sloth.lianjiadb.config.database.DBConfig$$EnhancerBySpringCGLIB$$bc1a0919] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2018-04-08 11:09:05.594 INFO 6081 --- [restartedMain] trationDelegate$BeanPostProcessorChecker : Bean 'DB' of type [class com.github.coolcool.sloth.lianjiadb.config.database.DB$$EnhancerBySpringCGLIB$$d7c7b6f7] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2018-04-08 11:09:05.624 INFO 6081 --- [restartedMain] com.zaxxer.hikari.HikariDataSource : HikariPool-0 - is starting. 2018-04-08 11:09:35.666 INFO 6081 --- [restartedMain] com.zaxxer.hikari.pool.HikariPool : HikariPool-0 - is closing down. 2018-04-08 11:09:40.673 WARN 6081 --- [restartedMain] o.s.b.f.s.DefaultListableBeanFactory : Bean creation exception on FactoryBean type check: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'houseindexMapper' defined in file [/Users/shawn/IdeaProjects/housedb/target/classes/com/github/coolcool/sloth/lianjiadb/mapper/HouseindexMapper.class]: Unsatisfied dependency expressed through bean property 'sqlSessionFactory': : Error creating bean with name 'sqlSessionFactory' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-0 - Connection is not available, request timed out after 30007ms.; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-0 - Connection is not available, request timed out after 30007ms.; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'sqlSessionFactory' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-0 - Connection is not available, request timed out after 30007ms.; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-0 - Connection is not available, request timed out after 30007ms. 2018-04-08 11:09:40.677 WARN 6081 --- [restartedMain] o.s.b.f.s.DefaultListableBeanFactory : Bean creation exception on FactoryBean type check: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'houseindexMapper' defined in file [/Users/shawn/IdeaProjects/housedb/target/classes/com/github/coolcool/sloth/lianjiadb/mapper/HouseindexMapper.class]: Unsatisfied dependency expressed through bean property 'sqlSessionFactory': : Error creating bean with name 'sqlSessionFactory': Requested bean is currently in creation: Is there an unresolvable circular reference?; nested exception is org.springframework.beans.factory.BeanCurrentlyInCreationException: Error creating bean with name 'sqlSessionFactory': Requested bean is currently in creation: Is there an unresolvable circular reference? 2018-04-08 11:09:40.678 INFO 6081 --- [restartedMain] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - is starting. 2018-04-08 11:10:10.682 INFO 6081 --- [restartedMain] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - is closing down. 2018-04-08 11:10:15.684 WARN 6081 --- [restartedMain] o.s.b.f.s.DefaultListableBeanFactory : Bean creation exception on FactoryBean type check: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'processMapper' defined in file [/Users/shawn/IdeaProjects/housedb/target/classes/com/github/coolcool/sloth/lianjiadb/mapper/ProcessMapper.class]: Unsatisfied dependency expressed through bean property 'sqlSessionFactory': : Error creating bean with name 'sqlSessionFactory' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-1 - Connection is not available, request timed out after 30003ms.; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-1 - Connection is not available, request timed out after 30003ms.; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'sqlSessionFactory' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-1 - Connection is not available, request timed out after 30003ms.; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-1 - Connection is not available, request timed out after 30003ms. 2018-04-08 11:10:15.690 INFO 6081 --- [restartedMain] com.zaxxer.hikari.HikariDataSource : HikariPool-2 - is starting. 2018-04-08 11:10:45.696 INFO 6081 --- [restartedMain] com.zaxxer.hikari.pool.HikariPool : HikariPool-2 - is closing down. 2018-04-08 11:10:50.701 WARN 6081 --- [restartedMain] o.s.b.f.s.DefaultListableBeanFactory : Bean creation exception on FactoryBean type check: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'processMapper' defined in file [/Users/shawn/IdeaProjects/housedb/target/classes/com/github/coolcool/sloth/lianjiadb/mapper/ProcessMapper.class]: Unsatisfied dependency expressed through bean property 'sqlSessionFactory': : Error creating bean with name 'sqlSessionFactory' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-2 - Connection is not available, request timed out after 30003ms.; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-2 - Connection is not available, request timed out after 30003ms.; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'sqlSessionFactory' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Unsatisfied dependency expressed through constructor argument with index 0 of type [javax.sql.DataSource]: : Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-2 - Connection is not available, request timed out after 30003ms.; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [com/github/coolcool/sloth/lianjiadb/config/database/DB.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Exception during pool initialization: HikariPool-2 - Connection is not available, request timed out after 30003ms. 2018-04-08 11:10:50.706 WARN 6081 --- [restartedMain] o.s.b.f.s.DefaultListableBeanFactory : Bean creation exception on FactoryBean type check: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'processMapper' defined in file [/Users/shawn/IdeaProjects/housedb/target/classes/com/github/coolcool/sloth/lianjiadb/mapper/ProcessMapper.class]: Unsatisfied dependency expressed through bean property 'sqlSessionFactory': : Error creating bean with name 'sqlSessionFactory': Requested bean is currently in creation: Is there an unresolvable circular reference?; nested exception is org.springframework.beans.factory.BeanCurrentlyInCreationException: Error creating bean with name 'sqlSessionFactory': Requested bean is currently in creation: Is there an unresolvable circular reference?

dee1024 commented 6 years ago

1.需要在你本地的Mysql建库,并且执行db目录下的 housedb.sql 2.在 /src/main/resources/application.properties 文件中,设置好对应的DB数据源 3.再次启动应用

chineping commented 6 years ago

@dee1024 也是一样的错误。

1、mysql > source housedb.sql # 提示执行成功,能看到area 表数据 2、src/main/resources/application.properties 配置好了对应数据库名和用户名, 也修改了 pom.xml 对应数据库 等 3、 重新 mvn clean packages -Dmaven.test.skip=true

4、再次启动应用java -jar housedb-20180719151928.jar

还是一样的错误信息。

housedb/deploy# java -jar housedb-20180719163043.jar

. _ _ /\ / '_ () \ \ \ \ ( ( )\ | ' | '| | ' \/ ` | \ \ \ \ \/ _)| |)| | | | | || (| | ) ) ) ) ' |__| .|| ||| |\, | / / / / =========|_|==============|__/=//// :: Spring Boot :: (v1.3.2.RELEASE)

2018-07-19 08:37:05.863 INFO 32382 --- [ restartedMain] c.g.c.sloth.lianjiadb.Application : Starting Application vV1.0 on 0f49bcd7f41d with PID 32382 (/dee1024_housedb/housedb/deploy/housedb-20180719163043.jar started by root in /dee1024_housedb/housedb/deploy) 2018-07-19 08:37:05.879 INFO 32382 --- [ restartedMain] c.g.c.sloth.lianjiadb.Application : No active profile set, falling back to default profiles: default 2018-07-19 08:37:06.180 INFO 32382 --- [ restartedMain] ationConfigEmbeddedWebApplicationContext : Refreshing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@310fcfe0: startup date [Thu Jul 19 08:37:06 UTC 2018]; root of context hierarchy 2018-07-19 08:37:06.999 INFO 32382 --- [kground-preinit] o.h.validator.internal.util.Version : HV000001: Hibernate Validator 5.2.2.Final 2018-07-19 08:37:10.232 INFO 32382 --- [ restartedMain] o.s.b.f.s.DefaultListableBeanFactory : Overriding bean definition for bean 'beanNameViewResolver' with a different definition: replacing [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/ErrorMvcAutoConfiguration$WhitelabelErrorViewConfiguration.class]] with [Root bean: class [null]; scope=; abstract=false; lazyInit=false; autowireMode=3; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=org.springframework.boot.autoconfigure.web.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter; factoryMethodName=beanNameViewResolver; initMethodName=null; destroyMethodName=(inferred); defined in class path resource [org/springframework/boot/autoconfigure/web/WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter.class]] 2018-07-19 08:37:11.380 INFO 32382 --- [ restartedMain] trationDelegate$BeanPostProcessorChecker : Bean 'DBConfig' of type [class com.github.coolcool.sloth.lianjiadb.config.database.DBConfig$$EnhancerBySpringCGLIB$$77e6bb7] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2018-07-19 08:37:11.385 INFO 32382 --- [ restartedMain] trationDelegate$BeanPostProcessorChecker : Bean 'DB' of type [class com.github.coolcool.sloth.lianjiadb.config.database.DB$$EnhancerBySpringCGLIB$$232c1995] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying) 2018-07-19 08:37:11.478 INFO 32382 --- [ restartedMain] com.zaxxer.hikari.HikariDataSource : HikariPool-0 - is starting.

959AI994 commented 10 months ago

请问,怎么启动数据分析啊,能不能发个数据?