SEAMAPD21 was created to enable fishery scientists and researchers to process raw image-based data at high volumes at a much quicker rate than current manual methods. This open-source dataset contains 90,000 annotations for 130 species of fish from two annual surveys done in 2018 and 2019 in strategic areas of the Gulf of Mexico using baited remote underwater video systems to assess reef fish populations.