ToDo Lists - May 2018

[baiqiushi]

Frontend (Twittermap)

• Right sidebar is hidden by scrollbar
• Pinmap left coner tool box needs to be updated or removed - Teyu raised PR still needs polish
• Popup Tweet content by talking to our own DB - Qiancheng raised PR, but needs DB version update, waiting for merged.
• Integrate multi-layered frontend feature - Teyu & Yang are working
• Show sentiment results for different maps (frontend design & backend query) - Tao is working

Bugs

• "Future timeout" exception (where and when is not known now, sometimes when starting the TM)
• Twittermap application.log is increasing fast when any client is alive
• Cloudberry berry.meta Cardinality number is not correct, 100M more than select count(*) from twitter.ds_tweet in AsterixDB. - Qiancheng is working
• Top50 Hash Tags on right side bar is not correct when in batch query slicing mode. (Different from SQL query directly from AsterixDB)

Features

• New 47000 size Sample Data (with user.pic_url attribute) is needed for Quick-Start guide environment
• HIVmap should be merged into our Github repository in some way
• Supporting "Geocell group by" queries with arbitrary scale (currently is fixed) as parameter in Cloudberry
• Randomly generate coordinate for each tweet which does not have coordinate value during geotagging preprocessing
• Materialized view management (It currently only creates new views without deleting views)
• HIVmap new demands from UCLA folks - Dengli is working
  • New time series results for each hot hash tag
  • New trending hash tags recently
• Profiling the whole workflow of Twittermap - Yishan is working on Frontend JS part.

Other

• Large display solution
  • One single big monitor (preferred)
  • Multiple small monitors
• Storage extension solution - Taewoo is working

Research

• Cache time series histogram in frontend - Ruiyao is working
• Show correlation map for different queries (keywords) - Yangguang is working
• Parallelize tweets ingestion process
• How to generate a good sample sets of points for pin-map - Liming is working
• Prefetch data in frontend

Related Projects

• WebGL - Using WebGL and GPU acceleration to enhance frontend rendering power. - Shuang's team is working.

• FPGA - Using FPGA to accelerate the image generation in Cloudberry tier. - Yu's Team is working.

  Potential Projects

  • A general service tool for people extracting data from a large Database, supporting predicates and schema customization
  • Not sure if possible, generalize Cloudberry to play a Query Planner role who treats any backend databases as blackbox and using some model to optimize the queries by leveraging statistics and indices to rewrite the query into queries to enhance Big Data Visualization user experiences.

[chenlica]

Will close it since it's obsolete.