medialab hyphe-traph issues

medialab / hyphe-traph

A Trie/Graph hybrid memory structure used by the Hyphe crawler to index pages & webentities.

MIT License

10 stars 2 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Add some lru caching of node lookups in the trie

#38 Yomguithereal opened 2 years ago
0
Recode Hyphe in rust and python3

#37 Yomguithereal closed 2 years ago
0
Investigate the respadon-sprint/theatre traph

#36 Yomguithereal closed 2 years ago
1
"Node has no right sibling" error

#35 boogheta closed 3 years ago
1
Unit testing edge case with prefix deletion

#33 Yomguithereal opened 5 years ago
0
The raw hashmap

#32 Yomguithereal opened 5 years ago
0
Variation nibble flag

#31 Yomguithereal opened 5 years ago
0
Add a paginate_webentity_pagelinks function

#30 boogheta closed 5 years ago
0
Test faster child_webentity collection

#29 boogheta closed 5 years ago
1
Compute stats about pagelinks weights

#28 boogheta closed 2 years ago
1
Optimize stem tail structure

#27 Yomguithereal opened 5 years ago
0
Adjust Hyphe's front to new traph capabilities

#26 boogheta closed 5 years ago
0
Do we add underlying number of pages on the trie's nodes?

#25 Yomguithereal opened 5 years ago
1
Optimize network generation (ram issues + time issues)

#24 boogheta opened 5 years ago
4
Cacher et renvoyer le nombre de pages par webentité à la génération du réseau

#23 boogheta closed 5 years ago
2
get_webentity_child_webentities takes forever on big entities

#22 boogheta closed 5 years ago
4
Add pagination capabilities to get_webentity_pages

#21 Yomguithereal closed 5 years ago
3
Add `max_depth` to `lru_trie.webentity_dfs_iter`

#20 Yomguithereal closed 5 years ago
0
CritBit & HAMT

#19 Yomguithereal opened 5 years ago
0
Try buffered memmap

#18 Yomguithereal opened 6 years ago
0
Store as metadata traph version number in data

#17 boogheta closed 5 years ago
1
Opti ram for get_links on huge corpus (lru_cache)

#16 boogheta closed 5 years ago
4
Also store source somehow in links

#15 Yomguithereal closed 5 years ago
4
Adding more debug info to iteration state

#14 Yomguithereal opened 6 years ago
0
Algorithmics: inlinks linked list apocalypse?

#13 Yomguithereal opened 7 years ago
3
Write some notes about the history of the structure's creation

#12 Yomguithereal closed 6 years ago
0
Investigate why indexing links from some WebEntities is way longer

#11 boogheta closed 7 years ago
0
Bug with get_webentity_pages

#10 boogheta closed 7 years ago
0
Optimize WebEntityCreationRules regexps

#9 boogheta closed 5 years ago
0
What to do when we recrawl a page

#34 boogheta opened 7 years ago
1
First constitute the tree in RAM when writing, then dump to file

#8 Yomguithereal closed 6 years ago
0
Does linked pages need to create WE?

#7 Yomguithereal closed 7 years ago
9
Should be possible to use 'c' type rather than 'B'

#6 Yomguithereal closed 7 years ago
2
Possible to store frequent nodes in a LRU cache in the instance for speedups

#5 Yomguithereal opened 7 years ago
0
Zero based issues

#4 Yomguithereal closed 7 years ago
1
How to perform DFS without quadratic stack size?

#3 Yomguithereal closed 7 years ago
1
Bit padding for struct (C reasons)

#2 Yomguithereal closed 7 years ago
0
Cursor hopping

#1 Yomguithereal closed 7 years ago
0