Closed bedeho closed 1 year ago
Monday, June 14th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage and label:carthage-low-prio
Runtime: Mokhtar progressing with Substrate v3 upgrades. Almost all pallets migrating. Issue tracked here:
Storage Node: no summary first week
Distributor Node: no summary first week
CLI: no summary first week
QN: Zeeshan completed Events mapping:
Ondrej completed checking if Hydra can work with v3 :
Pioneer: Fees implementation scope agreed and tracked here:
Atlas: Fees implementation scope agreed and tracked here:
The goal was to try to figure out the scope and critical path for Carthage release
Here is a brief summary of where we were:
Runtime: Ignazio -> fixes types Bedeho -> Review and merge Zeeshan -> enable QN testing Mokhtar to merge disabled bounties asap (13th June) Ignazio fixes bounty types MVP *(after merge) Luis to take on Pull #3485 and aim to replace bounties module for Carthage branch Ignazio to review PR to go to remove blog #3872
QN Zeeshan -> ETA of mappings in 1-2 days; QN buildable; after deletion will be done [❓⚠️ this point requires clarification from @ bedeho] Ondrej -> Content category system - #3574 Theo -> quick help with mappings on external resource identifiers from Ondrej, but to take care of it otherwise [this is not in scope for Carthage]
CLI Leszek should take care of it
Atlas and Pioneer
Recurring meetings Carthage all-hands held on Mondays and Wednesdays 10:30 CET.
Minting limits - Remains in scope. Tasks to handle this on QN and Atlas need to be re-opened [❓ ⚠️ this point requires clarification from @ bedeho] [Minting limits to be moved out of this ticket ❓⚠️ this point requires clarification from @ bedeho]
There are at least to three distinct parts to this question of NFT limits in the runtime:
UpdateGlobalNftLimitsProposal
update_global_nft_limit
check_nft_limits
We should turn them all off, I have reflected this here: https://github.com/Joystream/joystream/issues/3882
Zeeshan -> ETA of mappings in 1-2 days; QN buildable; after deletion will be done [❓⚠️ this point requires clarification from @ bedeho]
Not sure what the question is here. QN buildability does not depend on supporting deletions per say, it is about updating mappings to reflect latest event signatures as far as I know. Obviously the types library has to be updated to reflect the runtime as well.
Wednesday, June 15th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage and label:carthage-low-prio
⚠️ Some issues [potentially] were missing due to Asana-Github syncing outage; while some were marked as completed erroneously. Until next meeting pls refer to both Asana and Github for accurate representation of priorities.
Scope Update Freeze all limits removed from scope. Cannot use, cannot purchase, issuing NFT not sensitive to limits QN don't care, Atals don't care for mainnet, so as a result of above removed from Carthage and Mainnet.
NFTDestroyed
is in Rhodes - block this feature, no QN support reuquired. Label added post-mainnet
Runtime: Previously announced milestone for 16th Working local Node and QN -> unlikely but not impossible.
@ Mokhtar - continues with v3 substrate, nearing completion. ETA on new env is end of this week (optimistic), start of next week (realistic).
Ephesus -> Audit3 merge: complications of stale Audit3 branch caused overwhelming number of changes and conflicts. Decision was made to proceed with merge and handle the consequences as alternative scenarios do not seem to be feasible. [❓ ⚠️ Bedeho to confirm, as opening statement that we were "skipping it", not clear if for the meeting or as a task of focus for now]
@ Luis - pause on content pallet; already merged disabled Bounties v2;
Due date on content pallet benchmarking - 22nd June.
Next tasks for Luis: review cross-pallet calls:
@ Ignazio Fix bounty types ✅ done not on latest branch, need Bounty branch to be merged to Audit3.
Freezing runtime features : - https://github.com/Joystream/joystream/issues/3882 pushed back to next week ⏭️ in favour of new tasks (listed below)
Next focus on:
and
@ Bedeho is focussed on planning and reviewing issues:
Storage Node: no summary first week
Distributor Node: no summary first week
CLI: Awaiting Leszek (next week)
QN: Zeeshan Working on deletion, ETA is 1 or 2 days
Ondrej Working on metaprotocol contend directory, ETA today - decision was made to pause ⏸️, Keep as draft PR. channel cats -> removed, never needed by Atlas; video cats -> removed from runtime. Andy to focus on trying to keep schema the same to minimise work for Atlas team 🤞
Next: Channel agent permissions
Pioneer: Fees implementation scope agreed and tracked here:
Bounties deprecation:
Atlas: Fees implementation scope agreed and tracked here (🆕 fees are to be displayed in JOY, with <0.1 JOY for small fees. Smaller denomination will be displayed in tooltip. TBC as designs are now available for review:
Atlas designs will impact Pioneer designs. Scope of changes for existing pioneer component is TBC.
❗ Based on progress and remaining work, the Carthage release seem to be less risky and scope-heavy as initially anticipated, leaving some room to implement main-net focussed features, driven by FE apps demands. This point is however to be confirmed after Pioneer Regression testing is completed, subj to env availability.
This work will be undertaking by Builders WG.
Ticket to track:
⚠️ superseded by section "To-do Now" above
Runtime:
QN
CLI
Atlas
Pioneer
Recurring meetings Carthage all-hands held on Mondays and Wednesdays 10:30 CET.
Monday, June 20th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
ℹ️ Asana-github syncing was fixed.
Goal of getting runtime where Atlas, Pioneer, Argos and Colossus can be updated. ⚠️ New timeline: Node that has audit work done w new features disabled is going to be operational later this week. Original timeline of end of last week now extended.
ℹ️ Carthage will be Substrate_v3, based on Audit3; now we keep merging to Audit3 ℹ️
Action: ⚠️ @ all - make sure all PRs are labelled correctly, assign "carthage", and correct pallet label (or project-tokens for Ephesus work)
Separate zoom call will be done on #3823
🆕 Milestone added - Carthage Merged, end of this week.
➖
➕
Runtime:
@ Mokhtar
Focus on Content Directory conflicts with Iganzio
@ Ignazio
Runtime features freezing - done, in review now
Merge Master to Audit3 Focus on Content Directory conflicts with Mokhtar
@ Luis
2 PRs on CD video state bloat bond and channel state bload bond are ready. We need to merge before Mokhtar pulls it in. Can be merged now as code is ok (by Ignazio), only cosmetics left.
@ Leszek is back.
To review PRs from Luis and add to Audit3. To review the Merge that Ignazio picked. Merge to upgraded Master + Bounties v2 + Remove Blog pallet. To create issue on Withdrawal to Budget / Personal Acc to be done with same extrinsic
@ Bedeho
to add category to Asana for open PRs.
Runtime-PRs:
discussed the following group of PRs separately
@Lezek123 to review this group before "pair programming"
QN
Can build for Audit3 branch, but recent changes listed above would extend this task Ondrej and Zeeshan to progress with tasks from last meeting
Atlas and Pioneer
Left undiscussed, awaiting working QN for testing while working on implementation of fees.
⚠️ @ all - make sure all PRs are labelled correctly, assign "carthage", and correct pallet label (or project-tokens for Ephesus work) @ Bedeho - to add new category to Asana - PRs to cater for merges dependencies. @ Leszek - Review of PRs list 3809-3924; Create task on using same extrinsic for two diff balance transfers; @ Mokhtar - to work on Content Directory with Ignazio @ Ignazio - to work on Content Directory with Mokhtar @ Luis - continues benchmarking
Wednesday, June 22th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
Carthage
further to also not have any vesting releated treatment. This will have to be done for mainnet only. Testing should still try to verify manually that vesting works as expected, updated QA scope issue: https://github.com/Joystream/joystream/issues/3885ephesus
label to issue for future work. The only remaining issue from this scope was relating to supporting plain old creator withdrawals from the channel account, as that is still needed: https://github.com/Joystream/joystream/issues/3615.carthage-substrate-v3
, and retarget carthage
branch, which has the partial work on substrate upgrade and audit3 work. He will be blocked later by the fact that tests there are still not working, but he can pull in those fixes later when mokhar is done.Friday, June 24th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
➕ New tasks:
➖ Removed tasks:
🔴 Major complication
Runtime
Node is buildable, but requires update wrt types library changes introduced with Substrate3. Types is a blocker for buildable QN, and this cascades to blocking upstream apps Atlas and Pioneer do not need the running node to start applying adjustments There are 2 components to it: Mappings and Hydra / Indexer, with the latter to be fixed before mappings Changes Required: 1 - Types naming changed 2 - Type Aliases are no longer part of metadata; Verdict - automated types library is concluded to be less work and more robust compared to changing custom joystream types lib (also implication on the API, which defaults to auto-generated).
⚠️ New immediate target
Next milestone - to get the changes of types introduced asap.
For this:
1️⃣ @ Leszek to introduce new types library to upstream codebase to make membership / channel creation work 2️⃣ @ Klaudiusz and @ Theo to do changes to Atlas and Pioneer Respectively 3️⃣ @ Leszek makes routines to separate library (to a new one) 4️⃣ @ Ondrej Hydra type gen picked up by (Leszek skips QN) 5️⃣ @ Leszek to fix all the mappings and all the imports
@ Mokhtar opicks whatever makes most sense while Leszek is working on new types lib @ Ignazio #3809 go through all prs and fix them @ Luis Continues with bloat bonds for channel and videos (ETA for channels is end of day 24th)
QN @ Zeeshan - continues with deletion of channels and making sure QN builds @ Ondrej - continues with channel agent permissions
Monday, June 27th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
➕ New tasks:
@ Leszek to update typegen in Hydra, he will create a respective issue for tracking
➖ Removed tasks:
⏭️ Postponed tasks
Key Topic: Types Change
@ Leszek - completed the work to generate the types, there is a branch already. Needs to merge Mokhtar's branch to Carthage. The only thing that affects the Apps is that it does not have the FE constants, so some definitions are missing and need to be re-done on the apps side.
⚠️ When the apps can be updated wrt new types? - Once Leszek updates PR and do annotation, ETA today ⚠️
Discussed cross-pallet implications on weights @ Leszek shared video, @ Bedeho shared article from substrate.dev
Runtime
@ Mokhtar completed fixing content pallet tests @ Mokhtar is close to finish next iteration, pushed final changes -> final review pending by @ Leszek, ETA - today. @ Mokhtar - Fix build issues after upgrade, mainly benchmarks. ETA - tomorrow.
@ Luis - doing benchmarking, pushed. Fixed x2 state bloat bond tickets, merged with Audit3. To continue on benchmarking without changes till further notice
Runtime PRs
@ Ignazio to take care of the list below:
QN, Hydra
@ Zeeshan to update QN wrt to new types #3940 @ Leszek to update typegen in Hydra, he will create a respective issue for tracking @ Ondrej meta protocol, channel agent permission status will be updated on Wednesday due to absense.
Colossus, Argus, CLI @ Zeeshan to update CLI wrt to new types (after 3940), but needs Hydra upd for it. @ Leszek to upd Colossus to work with Audit3 + Fees (tasks merged on the meeting) @ Leszek to upd Argus, Colossus wrt to new types + API scripts (to make an example)
Atlas and Pioneer @ Klaudiusz to update Atlas @ Theo to update Pioneer [work to commence after 11th July]
To-do Next
@ Mokhtar is close to finish next iteration, pushed final changes -> final review pending by @ Leszek, ETA - today. @ Leszek - completed the work to generate the types, there is a branch already. Needs to merge Mokhtar's branch to Carthage. @ Leszek to update typegen in Hydra, he will create a respective issue for tracking @ Zeeshan to update QN wrt to new types #3940 @ Zeeshan to update CLI wrt to new types (after 3940), but needs Hydra upd for it. @ Leszek to upd Colossus to work with Audit3 + Fees (tasks merged on the meeting) @ Leszek to upd Argus, Colossus wrt to new types + API scripts (to make an example) @ Klaudiusz to update Atlas
Wednesday, June 29th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
➕ New tasks: none
➖ Removed tasks:
⏭️ Postponed tasks
To next week to make room for timely PRs review
Key Topic: Types Change Follow ups and Runtime Merges; Very big backlog of PRs, rather than issues prevents from accurate planning and became the priority overall.
Milestone of working local node is achieved 🦄 Community testing start date will be advised by @ Martin
Leszek and Zeeshan focussing on QN + Hydra types change implications.
@ Leszek - update Hydra Typegen - almost finished, tests already passing, what remains is to test with Joystream QN. @ Zeeshan - QN not testable yet, needs Hydra, will push the types to branch and Leszek will test.
@ Leszek - updated QN to work with typegen and after that will review runtime PRs. Namely,
@ Mokhtar - fix referendum benchmarks - today; fix build issues post substratev3 upgrade; carthage bootstrapping script (only memebrships)
@ Luis - to finish benchmarking, remove council blog pallet - to fix conflicts; Rebase video and channel bloat bond to Carthage. ETA on both to be merged is today.
@ Ignazio - finished on Channel transfer nonses. Will continue to review
low14 and low 15
Cross pallet extrinsic transaction
Unification of withdrawals to personal account
@ Leszek - update Hydra Typegen - almost finished, tests already passing, what remains is to test with Joystream QN.
@ Zeeshan - QN not testable yet, needs Hydra, will push the types to branch and Leszek will test.
@ Andy on channel agent premissions. Only 1PR reviewed = Curator moderation actions - To be merged after QN is buildable. Metaprotocol category system needs QN to test. Deletion of videos and channels
@ Zeeshan to take on - CLI to work in Carthage
Theo is away Klaudiusz waiting for QN and infra but some progress can be done with just working QN
Highest prio = runtime and QN core final features to be merged asap, full focus on PRs reviews and merging.
Monday, July 4th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
➕ New tasks:
➖ Removed tasks:
⏭️ Postponed tasks
Key Topic: Outstanding runtime PRs .
Milestone in focus - staging network. First community testing - ETA based on Martin, but need Runtime + QN + CLI updated. What do we need for it? - chain set up scripts to be fixed and QN merged.
PRs
TASKS
Focus: PRs dependant on buildable QN
Skipped
Skipped
Milestone in focus - staging network.
Wednesday, July 6th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
➕ New tasks: QN and CLI ticket:
➖ Removed tasks:
⏭️ Postponed tasks
Key Topic: Preparing for Carthage Staging Env
⭐ Carthage milestones ⭐ Staging network launch is nearing: node is ready for staging, QN almost ready with only tests left, ETA - end of this week. First community testing 18th July. Production release ETA is 1st August.
PRs
TASKS
Focus: make QN buildable (only tests left) and merge all PRs in review.
Buildable QN - @ Zeeshan
Migrating work on top of the merged branch and review Zeeshan's work - @ Ondrej
⚠️ Zeeshan is away early next week, will aim to finalize buildable qn and subsequent merges before he takes off
@ Leszek to check if it compiles, and if has any issues + Colossus: update to work w Audit3
Skipped
Milestone in focus - staging network.
For the lack of a better place to put this planning note, I am putting it in this thread.
@L3pereira noted that they believe it will take at least 3 weeks from today for them to complete remainder of benchmarking, but possibly much longer, and this is already some 6 weeks into it. This means we have to reorganize work on this, by parallelizing the effort, something which now should be possible as the core benchmarking infrastructure code has been completed.
@L3pereira
carthage
.carthage
label.@Lezek123
carthage_content_benchmark
based on current carthage
.carthage
label.@ignazio-bovo
carthage
label.Tuesday, July 12th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
➕ Added tasks:
➖ Removed tasks:
⏭️ Postponed tasks
Key Topic: Carthage Staging Env and testing timelines
⭐ Milestone of 1st staging playground for Carthage (QN + Argus + Colossus) - ETA tomorrow, 13th July.
⭐ Community Testing CLI based, orchestrated by Martin - ETA next week QA-epic for Carthage will be moved to Community Project55 once shared environment is up with help from @ Mokhtar; and Updated Types+ Fees are merged- ETA next week.
⭐ New types will be published for Atlas by @ Mokhtar tomorrow
The tests above will complement each other.
ℹ️ NB: runtime deep dive of the sync meeting is covered in the comment above
PRs
TASKS
⚠️ startup the chain scripts to get fixed @ Mokhtar
Carthage branhc runs fine, need Leszek's PR to get merged Mappings: up to date, bounties not done, but getting deprecated so not an issue Missing features (video categories, agent permissions) - not merged but not a blocker - ETA for merge - today
⭐ Milestone of 1st staging network QN + Argus + Colossus - ETA tomorrow
⭐ Pioneer "MVP" ready - end of next week
⭐ Atlas "MVP" ready - week after next one Types -done, tested with local env Fees calc - in progress -> end of this week for ETA Storage details (manual buckets) - to test and implement follow ups Bloat bond fees -> to do
Deploy playground, fix startup scripts, make FE apps work with new env and test the fees with community.
Thursday, July 15th, 2022.
Absent: None
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
Key Topic: Carthage Staging Env and testing timelines
⭐ Community Testing - moved to 25th July
⭐ Carthage Launch - moved to 12th Aug
⭐ Milestone of 1st staging playground for Carthage (QN + Argus + Colossus)
Pioneer MVP env -> Thursday next week QN: buildable Colossus Audit3 and Argus - some work remains to be done Faucet - small change to initial funds required, current amount (100) is not enough
⭐ Ephesus scope will be delivered after mainnet, as currently features are frozen on runtime.
Staging Env: Locally already works. Playground will be up soon, scripts on starting it up remains, but manually can be done. Remedy for staging to allocate funds to workers who run in the startup script, and hire leads via sudo.
PRs
TASKS Benchmarking - 3 weeks to go. (Suggestion to bump up the weights to higher, this way some bugs can be better exposed)
🆕 @ Leszek to create an issue on Bload Bonds for Channel accounts
Content category system, CLI + tests @ Andy Channel agent permissions - before the end of week @ Andy
CLI - @ Zeeshan Colossus Audit3 and Argus - some work remains to be done @ Zeeshan
Pioneer "MVP" ready - end of next week
Martin to create detailed testing plan
Deploy playground, fix startup scripts, make FE apps work with new env and test the fees with community.
Monday, July 25th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
Key Topic: FE apps readiness for testing
⭐ Pioneer - awaiting for denomination features merge tomorrow, and community testing to commence right after. ⭐ Atlas - everything will be working but not fully tested by Wednesday 27th expecting fully tested by end of this week.
NB: comments here represent first part of the meeting, before deep dive into Runtime part of the meeting.
"Currency PR"- NFT limits, fee for membership, change to value denominated based on market value of tokens, so fees will go up, + state bloat bonds; @ Leszek to review; Int tests to be run locally while CI checks are not working. Community testing will happen after merge. Feature on forum Polls - added to runtime, not required for Pioneer testing. By Thursday will get new playground with created channel account; id + event + QN for it.
@ Andy - off from Wednesday till next Monday.
@ Zeeshan - finalising tests and moving his focus to Youtube partnership programme. #3902 - moved to August 1st to commence.
Monday, Aug 1st, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
Key Topic: Carthage milestones update
⭐ Second Round of testing will be commenced on 18th Aug ⭐ Pioneer - QA commenced on Friday, some bugs reported, but nothing critical ⭐ Atlas - testing done, but to complete the suite, need staging env with multiple distributor nodes (to test allocation of storage buckets for channels creation and videos upl)
🆕 New Tasks
🔜 Postponed
NB: comments here represent first part of the meeting, before deep dive into Runtime part of the meeting.
For week one after next one:
Monday, Aug 8th, 2022.
Absent:
Tracked in Asana: Link to project timeline Tracked in Github: label:carthage
Key Topic: Carthage milestones update
⭐ Second Round of testing will be commenced on 18th Aug - still on track ⭐ Pioneer - QA: In progress. Details in relevant section. ⭐ Atlas - testing completed for all tickets, apart from Invitation lock testing.
🆕 New Tasks
🔜 Postponed
Only @ Leszek was present. Will address new issues described above and PR reviews earmarked for this week in Asana. @bedeho will speak to Luis on reviews status
QA status: Validators testing blocked by staging env, and Funding proposals by exposing max_limit. Members/ Council/ Forum/ Storage and Distributors are tested. Proposals; Invitation Lock; Vesting to be tested.
New task raised for post-mainnet raised
TODO as of 22 aug: aim for full benchmarking to be done by Tuesday Aug 30th.
mainnet
scope, even if My Payments
is not there.My Payments
will almost certainly not be included, since Atlas
team has solid backlog of work on enhancements and Gleev adjustments. Hence My Payments
related work should only start when other in-scope tasks are done.My payments
mainnet
Pioneer work.mainnet
Atlas work.Gleev
work is totally missing.mainnet
scope when he is done with last reviews and YPP work.YPP is just about done, end-to-end
Well I wouldn't say that's 100% accurate. While most of the tickets we've drafted for Atlas are done, there definitely needs to be some testing involved before we can release. There's also the operational side of things - making sure whoever is responsible on the JSG side has all the tools needed to run the program. So can easily look up activities that should be rewarded, can make payments, etc.
there definitely needs to be some testing involved before we can release
Good observations, added issue for getting a plan defined to begin with: https://github.com/Joystream/joystream/issues/4357
Integration ticket for YPP was put on hold and awaiting the rest of the YPP gleev adaptation.
There's no danger in including it to mainnet
scope as can be launched in a disabled/ hidden mode. This ticket encapsulate the work to make it so:
Let's take a view on this once the full scope is designed and estimated, which will be done end of day on October 17th, so we can discuss this on the meeting Oct 20th. The team will be busy with other mainnet
labelled issues before then.
Purpose
The purpose of this issue to track overall progress on the storage system in the
Carthage
release.┆Issue is synchronized with this Asana task by Unito