Closed rexwreyes closed 7 months ago
rexwreyes
commented
9 months ago Maybe i did not post correctly? Anyway my issue is this:
I have a table with roughly 20K items. i do a graphQL call using nextToken and cant ever seem to get all the items. It seems like the most i can get are around half or 10K. This is using pagination and a limit of 100.
stocaaro
commented
9 months ago Reviewing your code, I think there are issues here around setting multiple useState's which are triggering parallel useEffect. It looks like isLoading is intended to work like a lock preventing multiple updates from happening in parallel, but I don't think it's enforced. Looking at this, I would expect setAlertData to be called in ways where data is overwritten / lost during the load process.
To fix this, I would recommend performing the load inside of a single useEffect run without anything in the dependency array. Something like the following.
function App() {
const [alertData, setAlertData] = useState([]);
useEffect(() => {
const fetch = async (nextToken) => {
const variables = {
nextToken,
limit : 100,
}
return await API.graphql({ query: queries.listAlerts, variables });
}
const fetchAll = async () => {
let result = await fetch(undefined);
const allAlerts = [...result.data.listAlerts.items];
setAlertData(allAlerts); // We can assign alerts incrementally to see updates during the load.
while(result.data.listAlerts.nextToken) {
result = await fetch(result.data.listAlerts.nextToken);
allAlerts.push(...result.data.listAlerts.items);
setAlertData(allAlerts);
}
}
fetchAll();
}, []);
return <>...</>
}Can you try this approach to the data loading and let us know if you are able to retrieve all of the data you expect to?
rexwreyes
commented
9 months ago ok but i get the same results using the AWS App Sync console.
chrisbonifacio
commented
9 months ago @rexwreyes in that case, this doesn't seem to be an Amplify library issue but I'd be curious to see how you're querying in the AppSync console.
Please provide reproduction steps and/or some example queries and responses that demonstrate where the cutoff seems to happen during pagination.
cwomack
commented
9 months ago @rexwreyes, just want to confirm that this is happening on v4.x of Amplify (and not more recent versions like v5 or v6) right? If so, do you have the ability to upgrade to v6 and let us know if the issue persists? Additionally, if we could see your schema and associated auth modes, that would be very helpful! Thanks.
rexwreyes
commented
9 months ago yes, sorry. this is on v4, but I don't have the option of updating it as the client can't afford all that work ATM.
I got to the point of figuring out that I can use a sortKey and the index it builds will get all the records.
Before opening, please confirm:
JavaScript Framework
React
Amplify APIs
GraphQL API
Amplify Categories
api
Environment information
Describe the bug
I have a table with roughly 20K items. i do a graphQL call using nextToken and cant ever seem to get all the items. It seems like the most i can get are around half or 10K. This is using pagination and a limit of 100.
Expected behavior
To retrieve the data that im calling and know that it is there.
Reproduction steps
run the app
Code Snippet
Log output
aws-exports.js
No response
Manual configuration
No response
Additional configuration
No response
Mobile Device
No response
Mobile Operating System
No response
Mobile Browser
No response
Mobile Browser Version
No response
Additional information and screenshots
No response