Open MatthewRalston opened 6 months ago
g++???? Why not a++
Okay so e3eb1d8e39b6272351a562488900ff1db7ac1fac addresses the 3-tuple again (#124) during edge list and neighbor construction. A working data structure init process will create fundamental faculties for graph construction and traversal, regardless of implementation.
This is getting fleshed out in kmerdb/graph.py
What is needed for working data structure initialization? Why isn't it working?
The latest commit closes the issue of working neighbor structure implementation as a subtask. (also, it doesn't... ongoing.)
More on that later. While the neighbor structure "isn't" 'not working' right now, it appears fixed. For example, the neighbor structure pairings (the edge list from prior commits) have a k-1 overlap as expected. So simply put, the 'neighbors' functionality in this and prior commits appear to have a working 8-tuple that is added to the source by virtue of:
neighbors[k_id] = kmer.neighbors(cur, k_id, k)
I am using a simple method to initialize the edge list
The hidden tasks include
Key Question
What is needed for working data structure initialization? Why isn't it working?
The node and edge list and prioritization or sort strategy for edge representation, weights, multigraph and combination representation, orientation of edges, dual strandedness and .kdbg row metadata (non-int, but Boolean) (i.e. fast lookup) row metadata fields is not yet finalized.
Reverse walk
Walks files are just like path files, and primarily contain an ordering of edges. All walks are paths, but a walk may have a forward and reverse direction, and so all walks and their originating context (aka a .kdbg file) must either be minimal (all edges and a positioning id (i) only - a "retrospective " bool, a "solutional" bool (if the walk is said to be solutional from an assembly process associated from .kdbg version 1.0 0 or greater, a version number associated with the kmerdb release, the sha256 of the git release (on each edge yes), or expanded (retrospective, prospective, previous forks investigate and their node IDs)
A minimal walk file must also include all edges of the original context (a.k.a. all edges observed from the dataset(s) in the .kdbg header), marked with a retrospective bool, along with one or more copies of the same edge prospective bool = True
when representing a specific walk (not a minimal path, a single linear representation of edges, a sort order with no presumed origin id)
a walk, along with all previous walks (in chronological aka integer id, by reference, along with the sha256sum of the git release that produced the walk
Header metadata will have the source and the parameters in the header. And a walk id - (a sha256 of the walk) for an associated walk file, and walk name (given at "runtime" via CLI). May be 0 to represent unspecific or unqualified walk (origin unclear)
Related issues
126 #122
Key features
The latest commit closes the issue of working neighbor structure implementation as a subtask. (also, it doesn't... ongoing.)
More on that later. While the neighbor structure "isn't" 'not working' right now, it appears fixed. For example, the neighbor structure pairings (the edge list from prior commits) have a k-1 overlap as expected. So simply put, the 'neighbors' functionality in this and prior commits appear to have a working 8-tuple that is added to the source by virtue of: neighbors[k_id] = kmer.neighbors(cur, k_id, k)
I am using a simple method to initialize the edge list
- create the pair
- pass around the neighbor structure, AND edge list
- use pair to create "ordered" list of the key_id duple (path) (so .... etc.)
The hidden tasks include
- distinguish in data structure spec whether the nature of the pair is kmer to neighbor, or a "prospective" neighbor to k-mer, given by some prioritization parm set... etc. and thus a traversal approach
The neighbor structure 🌪️is manifested by particular kmer IDs🌬️, which may be accessed from kmer arrays loaded alongside the edge list during assembly.
A working pipeline would include all components of the workflow onto the next step but all commands are partial.
Key Comment... I guess i didn't know how valuable this is...
but there is no particular sort order. But what does that mean about your data and how its output. During printing, i tend to use a numerical index (1 : 4^k) or a lexical, provided order. Like from a graph or edge list, generated by virtue of a sequence from data input, where order should be known to the user. Provided order of the sequences in a fasta or fastq file determines the order of k-mers. However, the "neighbor" structure is generated sparsely, or incompletely. i.e. not all edges and orientations of nodes in the pair are represented in the posited neighbor structure, which is generated by k-mers from the dataset plus the 8 nearest neighbor k-mers. They are produced as such
--- k-1 mers (if k is 12, then the k-1-mer is 11bp long)
so this is the kdbg and neighbor structure of the input sequence data, and not the "total" or "theoretical" edge list from all 16,777,216 theoretically-derived k-mers. Nor is it the sequence data alone, but also all possible neighbor pairs from the sequence data alone, and therefore at least one or more paths through the neighbor structure to all relevant edges in a undirected graph producing weights.
The sort order is there to play with, its not task uno right now.
The
_shred
method returns a ordered list of kmer IDs. Assert two subsequent kmers in the list are neighborly (share a k-1 mer in common) or else do not increment that edge (it mustn't exist)