search

ga4gh / ga4gh-server

Reference implementation of the APIs defined in ga4gh-schemas. RETIRED 2018-01-24
http://ga4gh.org
Apache License 2.0
96 stars 93 forks source link

Provide parallel testing interface #1328

Open dcolligan opened 8 years ago

dcolligan commented 8 years ago

Not sure why, but currently the server tests are taking ~300s for 800 tests to run whereas a couple months ago the tests were taking ~100s for 600 tests to run.

now: https://travis-ci.org/ga4gh/server/builds/150061648 then: https://travis-ci.org/ga4gh/server/builds/120724174

We should investigate what is taking so long and look to optimize.

dcolligan commented 8 years ago

This run on my local machine was unexpectedly fast (usually it takes ~200s), but here's all the tests, sorted by running time:

After pip install nose-timer:

$ nosetests --with-timer --timer-top-n 1000
...
tests.end_to_end.test_gestalt.TestGestalt.testEndToEnd: 3.6686s
tests.unit.test_data_interface.TestInterfacingLayer.testGetFeature: 2.3951s
tests.end_to_end.test_repo_manager.RepoManagerEndToEndTest.testEndToEnd: 1.4535s
tests.end_to_end.test_oidc.TestOidc.testMultipleOidcClients: 1.2714s
tests.unit.test_utils.TestTimeout.testTimeoutException: 1.0037s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchReads: 0.9064s
tests.unit.test_simulated_stack.TestSimulatedStack.testListReferenceBasesPaging: 0.7362s
tests.end_to_end.test_oidc.TestOidc.testOidc: 0.6141s
tests.end_to_end.test_sequenceAnnotations.TestSequenceAnnotations.testSearchFeatures: 0.5799s
tests.unit.test_repo_manager.TestAddOntology.testWithSameName: 0.5604s
tests.unit.test_repo_manager.TestVerify.testVerify: 0.5228s
tests.unit.test_repo_manager.TestDuplicateNameDelete.testVariantSetDelete: 0.5076s
tests.unit.test_repo_manager.TestDuplicateNameDelete.testFeatureSetDelete: 0.4912s
tests.unit.test_repo_manager.TestDuplicateNameDelete.testReadGroupSetDelete: 0.4892s
tests.end_to_end.test_server_start.TestServerStart.testServerStart: 0.4795s
tests.unit.test_repo_manager.TestAddAnnotatedVariantSet.testAnnotationsNoOntology: 0.4765s
tests.unit.test_repo_manager.TestRemoveFeatureSet.testRemoveFeatureSet: 0.4576s
tests.end_to_end.test_oidc.TestOidc.testOidcBadLoginKey: 0.3744s
tests.unit.test_client.TestExhaustiveListingsLocal.testAllReads: 0.3684s
tests.unit.test_client.TestExhaustiveListingsHttp.testAllReads: 0.3634s
tests.unit.test_repo_manager.TestAddFeatureSet.testAddFeatureSet: 0.3183s
tests.unit.test_client.TestExhaustiveListingsLocal.testAllVariants: 0.3089s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchVariantSets: 0.2929s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchReferences: 0.2925s
tests.unit.test_converters.TestSamConverter.testSamConversion: 0.2670s
tests.unit.test_repo_manager.TestAddAnnotatedVariantSet.testAnnotations: 0.2656s
tests.unit.test_repo_manager.TestAddAnnotatedVariantSet.testNoAnnotations: 0.2635s
tests.unit.test_data_interface.TestInterfacingLayer.testGetRnaQuantificationSet: 0.2541s
tests.unit.test_repo_manager.TestAddFeatureSet.testAddFeatureSetBadOntology: 0.2529s
tests.unit.test_repo_manager.TestAddFeatureSet.testAddFeatureSetBadReferenceSet: 0.2528s
tests.unit.test_repo_manager.TestAddFeatureSet.testAddFeatureSetNoReferenceSet: 0.2522s
tests.unit.test_repo_manager.TestAddFeatureSet.testAddFeatureSetNoOntology: 0.2521s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchReadGroupSets: 0.2476s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchVariantAnnotationSets: 0.2389s
tests.unit.test_simulated_stack.TestSimulatedStack.testCallSetsSearch: 0.2283s
tests.unit.test_variant_annotations.TestHtslibVariantAnnotationSet.testThousandGenomesAnnotation: 0.2209s
tests.unit.test_repo_manager.TestAddAnnotatedVariantSet.testAnnotationsBadOntology: 0.1955s
tests.datadriven.test_variant_annotations.testVariantAnnotationSets: 0.1914s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetCallSet: 0.1877s
tests.unit.test_client.TestExhaustiveListingsHttp.testAllVariants: 0.1818s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchCallSets: 0.1813s
tests.unit.test_repo_manager.TestRemoveOntology.testDefaults: 0.1711s
tests.unit.test_repo_manager.TestAddOntology.testDefaults: 0.1601s
tests.unit.test_repo_manager.TestAddOntology.testWithName: 0.1588s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetBioSample: 0.1377s
tests.unit.test_simulated_stack.TestSimulatedStack.testReads: 0.1353s
tests.unit.test_repo_manager.TestAddVariantSet.testAddVariantSetWithSameName: 0.1319s
tests.unit.test_data_interface.TestInterfacingLayer.testGetVariantSet: 0.1293s
tests.unit.test_data_interface.TestInterfacingLayer.testGetVariantAnnotationSet: 0.1232s
tests.unit.test_data_interface.TestInterfacingLayer.testGetReadGroupSet: 0.1214s
tests.unit.test_data_interface.TestInterfacingLayer.testGetReadGroup: 0.1210s
tests.unit.test_cli.TestClientArguments.testBioSamplesGetArguments: 0.1199s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchReferenceSets: 0.1193s
tests.unit.test_data_interface.TestInterfacingLayer.testGetCallSet: 0.1179s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchFeatureSets: 0.1152s
tests.unit.test_backends.TestSqlRepoTestData.testDatasets: 0.1111s
tests.unit.test_data_interface.TestInterfacingLayer.testGetReference: 0.1104s
tests.end_to_end.test_oidc.TestOidc.testOidcBadLoginPassword: 0.1100s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchRnaQuantifications: 0.1097s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchRnaQuantificationSets: 0.1096s
tests.unit.test_variant_annotations.TestHtslibVariantAnnotationSet.testConvertLocation: 0.1096s
tests.unit.test_variant_annotations.TestHtslibVariantAnnotationSet.testConvertLocationHgvsP: 0.1094s
tests.unit.test_variant_annotations.TestHtslibVariantAnnotationSet.testConvertLocationHgvsC: 0.1094s
tests.unit.test_data_interface.TestInterfacingLayer.testGetRnaQuantification: 0.1093s
tests.unit.test_variant_annotations.TestHtslibVariantAnnotationSet.testHashVariantAnnotation: 0.1093s
tests.unit.test_data_interface.TestInterfacingLayer.testGetReferenceSet: 0.1090s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchDatasets: 0.1085s
tests.unit.test_data_interface.TestInterfacingLayer.testGetBioSample: 0.1083s
tests.unit.test_variant_annotations.TestHtslibVariantAnnotationSet.testAddLocations: 0.1083s
tests.unit.test_data_interface.TestInterfacingLayer.testGetVariant: 0.1082s
tests.unit.test_data_interface.TestInterfacingLayer.testGetFeatureSet: 0.1081s
tests.unit.test_variant_annotations.TestHtslibVariantAnnotationSet.testGetTranscriptEffectId: 0.1078s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchIndividuals: 0.1076s
tests.unit.test_data_interface.TestInterfacingLayer.testGetDataset: 0.1074s
tests.unit.test_data_interface.TestInterfacingLayer.testGetExpressionLevel: 0.1070s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchBioSamples: 0.1069s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchVariantAnnotations: 0.1065s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchExpressionLevels: 0.1065s
tests.unit.test_backends.TestSqlRepoTestData.testReferenceSets: 0.1064s
tests.unit.test_data_interface.TestInterfacingLayer.testGetIndividual: 0.1061s
tests.unit.test_data_interface.TestInterfacingLayer.testSearchVariants: 0.1058s
tests.unit.test_simulated_stack.TestSimulatedStack.testVariantAnnotationsSearch: 0.1026s
tests.unit.test_repo_manager.TestAddReadGroupSet.testAddReadGroupSetWithSameName: 0.0951s
tests.unit.test_gff3_parser.TestGff3ParserOnTypicalFile.testAllChildrenFeaturesArePresentInSet: 0.0869s
tests.unit.test_gff3_parser.TestGff3ParserOnTypicalFile.testRootFeaturesHaveNoParents: 0.0864s
tests.unit.test_gff3_parser.TestGff3ParserOnTypicalFile.testFileParsedHasSomeRootFeatures: 0.0857s
tests.unit.test_gff3_parser.TestGff3ParserOnTypicalFile.testInvalidFeatureNameKeyQueryFails: 0.0850s
tests.unit.test_gff3_parser.TestGff3ParserOnTypicalFile.testSomeFeatureIsWellFormed: 0.0849s
tests.unit.test_gff3_parser.TestGff3ParserOnTypicalFile.testAllFeaturesContainAllRootFeatures: 0.0843s
tests.unit.test_repo_manager.TestRemoveVariantSet.testWithForce: 0.0809s
tests.unit.test_repo_manager.TestInvalidVariantIndexFile.testNonexistentIndexFile: 0.0796s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetIndividual: 0.0780s
tests.unit.test_repo_manager.TestInvalidVariantIndexFile.testIndexFileNotAnIndexFile: 0.0771s
tests.unit.test_intervals.TestIntervalIterator.testOutsideRange: 0.0770s
tests.unit.test_simulated_stack.TestSimulatedStack.testListReferenceBases: 0.0761s
tests.unit.test_client.TestExhaustiveListingsLocal.testAllReadGroupSets: 0.0758s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetReadGroup: 0.0746s
tests.end_to_end.test_sequenceAnnotations.TestSequenceAnnotations.testSearchFeaturesByGeneSymbol: 0.0741s
tests.unit.test_repo_manager.TestRemoveDataset.testContainsReadGroupSet: 0.0707s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetReference: 0.0702s
tests.unit.test_repo_manager.TestAddVariantSet.testLocalFilesWithIndexes: 0.0689s
tests.unit.test_repo_manager.TestRemoveReadGroupSet.testWithForce: 0.0682s
tests.unit.test_repo_manager.TestAddVariantSet.testDefaultsLocalFiles: 0.0677s
tests.unit.test_repo_manager.TestAddReferenceSet.testWithSameName: 0.0669s
tests.unit.test_repo_manager.TestAddVariantSet.testDefaultsLocalDirectory: 0.0654s
tests.unit.test_simulated_stack.TestSimulatedStack.testReferencesSearchFilters: 0.0635s
tests.unit.test_repo_manager.TestAddVariantSet.testMissingReferenceSet: 0.0609s
tests.end_to_end.test_repo_manager.RepoManagerEndToEndTest.testForce: 0.0607s
tests.unit.test_client.TestExhaustiveListingsHttp.testAllReadGroupSets: 0.0604s
tests.unit.test_repo_manager.TestAddReadGroupSet.testLocalFileWithName: 0.0593s
tests.unit.test_repo_manager.TestAddReadGroupSet.testLocalFileWithIndex: 0.0593s
tests.unit.test_simulated_stack.TestSimulatedStack.testReadGroupSetsSearch: 0.0586s
tests.unit.test_intervals.TestIntervalIterator.testRandomStarts: 0.0575s
tests.datadriven.test_variants.testVariantSets: 0.0572s
tests.unit.test_repo_manager.TestAddReadGroupSet.testDefaultsLocalFile: 0.0565s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetVariant: 0.0555s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetVariantSet: 0.0549s
tests.unit.test_repo_manager.TestAddReadGroupSet.testMissingReferenceSet: 0.0541s
tests.unit.test_repo_manager.TestInvalidReadGroupSetIndexFile.testIndexFileNotAnIndexFile: 0.0535s
tests.unit.test_repo_manager.TestInvalidReadGroupSetIndexFile.testNonexistentIndexFile: 0.0534s
tests.unit.test_repo_manager.TestRemoveDataset.testEmptyDatasetForce: 0.0532s
tests.unit.test_repo_manager.TestAddReadGroupSet.testUrlWithMissingIndex: 0.0522s
tests.unit.test_repo_manager.TestAddVariantSet.testMissingDataset: 0.0519s
tests.unit.test_repo_manager.TestAddVariantSet.testUrlWithMissingIndex: 0.0516s
tests.unit.test_schemas.SearchResponseBuilderTest.testPageSizeOverflow: 0.0512s
tests.unit.test_repo_manager.TestAddReadGroupSet.testMissingDataset: 0.0508s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetFeatureSet: 0.0467s
tests.unit.test_intervals.TestIntervalIterator.testRandomIntervals: 0.0449s
tests.unit.test_repo_manager.TestRemoveReferenceSet.testDefaults: 0.0441s
tests.unit.test_intervals.TestIntervalIterator.testRandomEnds: 0.0426s
tests.unit.test_repo_manager.TestAddDataset.testSameName: 0.0417s
tests.unit.test_repo_manager.TestAddReferenceSet.testDefaults: 0.0353s
tests.unit.test_simulated_stack.TestSimulatedStack.testVariantsSearch: 0.0343s
tests.unit.test_repo_manager.TestAddDataset.testDefaults: 0.0331s
tests.unit.test_repo_manager.TestAddReferenceSet.testWithName: 0.0326s
tests.unit.test_simulated_stack.TestSimulatedStack.testListReferenceBasesErrors: 0.0317s
tests.unit.test_repo_manager.TestAddOntology.testMissingFile: 0.0309s
tests.end_to_end.test_sequenceAnnotations.TestSequenceAnnotations.testSearchFeaturesByName: 0.0307s
tests.unit.test_views.TestFrontend.testRouteReferences: 0.0302s
tests.unit.test_repo_manager.TestAddOntology.testNonOboBinaryFile: 0.0299s
tests.unit.test_views.TestFrontend.testRouteReads: 0.0299s
tests.unit.test_repo_manager.TestAddOntology.testNonOboTextFile: 0.0296s
tests.unit.test_simulator.TestSimulatedVariantAnnotationSet.testCreation: 0.0296s
tests.unit.test_views.TestFrontend.testRouteVariants: 0.0283s
tests.unit.test_oidc.TestFrontendOidc.testKeyParamAllowsIndex: 0.0277s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetVariantAnnotationSet: 0.0261s
tests.unit.test_simulated_stack.TestSimulatedStack.testReferenceSetsSearchFilters: 0.0246s
tests.unit.test_simulated_stack.TestSimulatedStack.testVariantSetsSearch: 0.0229s
tests.unit.test_simulated_stack.TestSimulatedStack.testReferencesSearch: 0.0224s
tests.unit.test_views.TestFrontend.testCors: 0.0219s
tests.unit.test_intervals.TestIntervalIterator.testFullInterval: 0.0212s
tests.unit.test_intervals.TestIntervalIterator.testStartInGap: 0.0193s
tests.unit.test_simulated_stack.TestSimulatedStack.testBioSamplesSearch: 0.0191s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetFeature: 0.0189s
tests.unit.test_simulated_stack.TestSimulatedStack.testReadsMultipleReadGroupSets: 0.0189s
tests.unit.test_faultydata.TestInconsistentCallSetId.testInstantiation: 0.0176s
tests.unit.test_simulated_stack.TestSimulatedStack.testBioSamplesFromReadGroupSets: 0.0174s
tests.unit.test_simulated_stack.TestSimulatedStack.testFeatureSetsSearch: 0.0171s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetReferenceSet: 0.0171s
tests.unit.test_cli.TestClientArguments.testReadGroupGetArguments: 0.0166s
tests.unit.test_cli.TestClientArguments.testIndividualsGetArguments: 0.0164s
tests.unit.test_cli.TestClientArguments.testRnaQuantificationGetArguments: 0.0162s
tests.unit.test_cli.TestClientArguments.testVariantAnnotationsSearch: 0.0159s
tests.unit.test_cli.TestClientArguments.testReferenceBasesListArguments: 0.0159s
tests.unit.test_cli.TestClientArguments.testFeatureSetsGet: 0.0158s
tests.unit.test_cli.TestClientArguments.testVariantsSearchArguments: 0.0157s
tests.unit.test_cli.TestClientArguments.testParseFailure: 0.0156s
tests.unit.test_cli.TestClientArguments.testDatasetsGetArguments: 0.0154s
tests.unit.test_cli.TestClientArguments.testReadGroupSetsSearchArguments: 0.0150s
tests.unit.test_simulated_stack.TestSimulatedStack.testGetDataset: 0.0150s
tests.unit.test_cli.TestClientArguments.testOutputFormat: 0.0148s
tests.unit.test_cli.TestClientArguments.testVariantSetsSearchArguments: 0.0148s
tests.unit.test_cli.TestClientArguments.testReferenceSetsSearchArguments: 0.0147s
tests.unit.test_cli.TestClientArguments.testVariationAnnotationSetsGet: 0.0146s
tests.unit.test_cli.TestClientArguments.testRnaQuantificationSetsGet: 0.0145s
tests.unit.test_cli.TestClientArguments.testVariantGetArguments: 0.0144s
tests.unit.test_cli.TestClientArguments.testDatasetsSearchArguments: 0.0144s
tests.unit.test_cli.TestClientArguments.testFeatureSetsSearch: 0.0143s
tests.unit.test_cli.TestClientArguments.testReferenceGetArguments: 0.0143s
tests.unit.test_cli.TestClientArguments.testCallSetsSearchArguments: 0.0142s
tests.unit.test_cli.TestClientArguments.testExpressionLevelsGet: 0.0142s
tests.unit.test_cli.TestClientArguments.testRnaQuantificationSetsSearch: 0.0141s
tests.unit.test_cli.TestClientArguments.testVariationAnnotationSetsSearch: 0.0141s
tests.unit.test_cli.TestClientArguments.testReferenceSetGetArguments: 0.0140s
tests.unit.test_cli.TestClientArguments.testReadsSearchArguments: 0.0140s
tests.unit.test_cli.TestClientArguments.testReadGroupSetGetArguments: 0.0139s
tests.unit.test_cli.TestClientArguments.testRnaQuantificationSearchArguments: 0.0139s
tests.unit.test_cli.TestClientArguments.testVariantSetsGet: 0.0139s
tests.unit.test_cli.TestClientArguments.testCallSetGetArguments: 0.0138s
tests.unit.test_cli.TestClientArguments.testReferencesSearchArguments: 0.0138s
tests.unit.test_cli.TestClientArguments.testFeaturesGet: 0.0138s
tests.unit.test_cli.TestClientArguments.testFeaturesSearch: 0.0137s
tests.unit.test_cli.TestClientArguments.testIndividualsSearchArguments: 0.0137s
tests.unit.test_cli.TestClientArguments.testExpressionLevelSearchArguments: 0.0137s
tests.unit.test_cli.TestClientArguments.testBioSamplesSearchArguments: 0.0137s
tests.unit.test_datarepo.TestDataRepoVersion.testRightVersion: 0.0131s
tests.unit.test_client.TestPagingLocal.testPageSize1: 0.0128s
tests.unit.test_simulated_stack.TestSimulatedStack.testFeaturesSearch: 0.0121s
tests.unit.test_client.TestPagingHttp.testPageSize1: 0.0116s
tests.unit.test_datarepo.TestDataRepoVersion.testWrongVersion: 0.0115s
tests.unit.test_protocol_errors.TestFrontendErrors.testPageSize: 0.0111s
tests.unit.test_client.TestExhaustiveListingsHttp.testAllVariantSets: 0.0110s
tests.unit.test_cli.TestRepoManagerCli.testAddVariantSetWithIndexFiles: 0.0110s
tests.unit.test_cli.TestRepoManagerCli.testRemoveReadGroupSet: 0.0110s
tests.unit.test_exceptions.TestExceptionConsistency.testGetExceptionClass: 0.0109s
tests.unit.test_faultydata.TestInconsistentMetaData.testInstantiation: 0.0105s
tests.unit.test_client.TestExhaustiveListingsLocal.testAllVariantSets: 0.0104s
tests.unit.test_cli.TestRepoManagerCli.testAddDataset: 0.0104s
tests.unit.test_cli.TestRepoManagerCli.testAddReadGroupSet: 0.0096s
tests.unit.test_cli.TestRepoManagerCli.testAddBioSample: 0.0095s
tests.unit.test_cli.TestRepoManagerCli.testRemoveBioSample: 0.0093s
tests.unit.test_client.TestPagingLocal.testPageSize2: 0.0092s
tests.unit.test_cli.TestRepoManagerCli.testAddIndividual: 0.0090s
tests.unit.test_cli.TestRepoManagerCli.testAddVariantSet: 0.0090s
tests.unit.test_cli.TestRepoManagerCli.testVerify: 0.0089s
tests.unit.test_cli.TestRepoManagerCli.testAddReferenceSet: 0.0089s
tests.unit.test_cli.TestRepoManagerCli.testRemoveVariantSet: 0.0088s
tests.unit.test_client.TestPagingHttp.testPageSize2: 0.0088s
tests.unit.test_cli.TestRepoManagerCli.testInit: 0.0088s
tests.unit.test_cli.TestRepoManagerCli.testRemoveReferenceSet: 0.0088s
tests.unit.test_cli.TestRepoManagerCli.testRemoveDataset: 0.0087s
tests.unit.test_cli.TestRepoManagerCli.testAddReadGroupSetWithIndexFile: 0.0087s
tests.unit.test_cli.TestRepoManagerCli.testRemoveIndividual: 0.0086s
tests.unit.test_cli.TestRepoManagerCli.testAddOntology: 0.0083s
tests.unit.test_simulated_stack.TestSimulatedStack.testBioSamplesFromCallSets: 0.0083s
tests.unit.test_cli.TestRepoManagerCli.testRemoveOntology: 0.0082s
tests.unit.test_client.TestExhaustiveListingsLocal.testAllReferences: 0.0082s
tests.unit.test_cli.TestRepoManagerCli.testList: 0.0081s
tests.unit.test_client.TestPagingHttp.testPageSize3: 0.0080s
tests.unit.test_client.TestPagingLocal.testPageSize3: 0.0080s
tests.unit.test_client.TestExhaustiveListingsHttp.testAllReferences: 0.0075s
tests.unit.test_simulator.TestSimulatedVariantSet.testConsistency: 0.0068s
tests.unit.test_views.TestFrontend.test404sReturnJson: 0.0064s
tests.unit.test_client.TestPagingLocal.testPageSizeAlmostListLength: 0.0062s
tests.unit.test_client.TestPagingLocal.testPageSizeListLength: 0.0062s
tests.unit.test_response_generators.TestReadsGenerator.testReadGroupEmpty: 0.0060s
tests.unit.test_client.TestPagingHttp.testPageSizeListLength: 0.0060s
tests.unit.test_client.TestPagingHttp.testPageSizeAlmostListLength: 0.0060s
tests.unit.test_client.TestPagingLocal.testDefaultPageSize: 0.0058s
tests.unit.test_client.TestPagingHttp.testDefaultPageSize: 0.0057s
tests.unit.test_oidc.TestFrontendOidc.testOidcCallbackBadState: 0.0053s
tests.unit.test_schemas.SearchResponseBuilderTest.testMaxBufferSizeOverridesPageSize: 0.0052s
tests.unit.test_schemas.SearchResponseBuilderTest.testPageSizeExactFill: 0.0052s
tests.unit.test_views.TestFrontend.testRouteCallSets: 0.0048s
tests.unit.test_oidc.TestFrontendOidc.testOidcCallbackSucceeds: 0.0047s
tests.unit.test_views.TestFrontend.testVariantsSearch: 0.0045s
tests.unit.test_oidc.TestFrontendOidc.testSessionKeyAllowsIndex: 0.0045s
tests.unit.test_views.TestFrontend.testGetVariantSet: 0.0044s
tests.unit.test_simulator.TestSimulatedVariantSet.testSelfConsistent: 0.0043s
tests.unit.test_simulated_stack.TestSimulatedStack.testVariantAnnotationSetsSearch: 0.0043s
tests.unit.test_response_generators.TestReadsGenerator.testReadGroupOneRead: 0.0042s
tests.unit.test_faultydata.TestOverlappingVcfVariants.testInstantiation: 0.0042s
tests.unit.test_simulated_stack.TestSimulatedStack.testSearchIndividuals: 0.0041s
tests.datadriven.test_reads.testReads: 0.0040s
tests.unit.test_views.TestFrontend.testGetDataset: 0.0040s
tests.unit.test_client.TestExhaustiveListingsHttp.testAllRnaQuantificationSets: 0.0040s
tests.unit.test_views.TestFrontend.testReadsSearch: 0.0040s
tests.unit.test_oidc.TestFrontendOidc.testOidcCallbackBadNonce: 0.0039s
tests.unit.test_oidc.TestFrontendOidc.testOidcCallbackBadTokenResponse: 0.0039s
tests.unit.test_response_generators.TestReadsGenerator.testNoReadGroupsNotSupported: 0.0039s
tests.unit.test_response_generators.TestReadsGenerator.testReadGroupTwoReads: 0.0039s
tests.unit.test_response_generators.TestVariantsGenerator.testVariantSetEmpty: 0.0039s
tests.unit.test_simulated_stack.TestSimulatedStack.testUnsupportedReadOperations: 0.0039s
tests.unit.test_response_generators.TestVariantsGenerator.testVariantSetTwoVariants: 0.0039s
tests.unit.test_client.TestExhaustiveListingsLocal.testAllRnaQuantificationSets: 0.0039s
tests.unit.test_datasets.TestDatasets.testToProtocolElement: 0.0038s
tests.unit.test_response_generators.TestVariantsGenerator.testNonexistentVariantSet: 0.0038s
tests.unit.test_gff3_parser.TestGff3ParserOnSacCerFile.testAllFeaturesContainAllRootFeatures: 0.0038s
tests.unit.test_response_generators.TestVariantsGenerator.testVariantSetOneVariant: 0.0037s
tests.unit.test_response_generators.TestReadsGenerator.testNonexistentReadGroup: 0.0036s
tests.unit.test_gff3_parser.TestGff3ParserOnSacCerFile.testAllChildrenFeaturesArePresentInSet: 0.0036s
tests.unit.test_views.TestFrontend.testCallSetsSearch: 0.0034s
tests.unit.test_simulator.TestSimulatedVariantSet.testGetVariants: 0.0034s
tests.unit.test_gff3_parser.TestGff3ParserOnSacCerFile.testInvalidFeatureNameKeyQueryFails: 0.0033s
tests.unit.test_oidc.TestFrontendOidc.testInvalidSessionKeyRedirects: 0.0033s
tests.unit.test_gff3_parser.TestGff3ParserOnSacCerFile.testFileParsedHasSomeRootFeatures: 0.0032s
tests.unit.test_gff3_parser.TestGff3ParserOnSacCerFile.testRootFeaturesHaveNoParents: 0.0032s
tests.unit.test_gff3_parser.TestGff3ParserOnSacCerFile.testSomeFeatureIsWellFormed: 0.0032s
tests.unit.test_schemas.SearchResponseBuilderTest.testIntegrity: 0.0032s
tests.unit.test_views.TestFrontend.testExpressionLevelsSearch: 0.0029s
tests.unit.test_simulated_stack.TestSimulatedStack.testReferenceSetsSearch: 0.0028s
tests.unit.test_views.TestFrontend.testRouteIndex: 0.0027s
tests.unit.test_simulator.TestSimulatedReadGroupSet.testCreation: 0.0027s
tests.unit.test_imports.TestImports.testLayering: 0.0025s
tests.unit.test_gff3_parser.TestGff3ParserOnDiscontinuousFeatureFile.testAllFeaturesContainAllRootFeatures: 0.0024s
tests.unit.test_views.TestFrontend.testGetVariant: 0.0024s
tests.unit.test_views.TestFrontend.testGetReadGroupSet: 0.0023s
tests.unit.test_client.TestExhaustiveListingsLocal.testAllDatasets: 0.0023s
tests.unit.test_oidc.TestFrontendOidc.testInvalidKeyParamIsUnauthorized: 0.0022s
tests.unit.test_views.TestFrontend.testSearchUnmappedReads: 0.0022s
tests.unit.test_gff3_parser.TestGff3ParserOnDiscontinuousFeatureFile.testInvalidFeatureNameKeyQueryFails: 0.0022s
tests.unit.test_oidc.TestFrontendOidc.testAuthorizationRequest: 0.0022s
tests.unit.test_views.TestFrontend.testGetRnaQuantificationSet: 0.0021s
tests.unit.test_gff3_parser.TestGff3ParserOnDiscontinuousFeatureFile.testDiscontinuousFeature: 0.0021s
tests.unit.test_gff3_parser.TestGff3ParserOnDiscontinuousFeatureFile.testSomeFeatureIsWellFormed: 0.0020s
tests.unit.test_gff3_parser.TestGff3ParserOnDiscontinuousFeatureFile.testAllChildrenFeaturesArePresentInSet: 0.0020s
tests.unit.test_client.TestExhaustiveListingsHttp.testAllReferenceSets: 0.0020s
tests.unit.test_views.TestFrontend.testVariantSetsSearch: 0.0020s
tests.unit.test_gff3_parser.TestGff3ParserOnDiscontinuousFeatureFile.testFileParsedHasSomeRootFeatures: 0.0019s
tests.unit.test_simulator.TestSimulatedVariantSet.testConstruction: 0.0019s
tests.unit.test_gff3_parser.TestGff3ParserOnDiscontinuousFeatureFile.testRootFeaturesHaveNoParents: 0.0019s
tests.unit.test_simulated_stack.TestSimulatedStack.testDatasetsSearch: 0.0019s
tests.unit.test_client.TestExhaustiveListingsLocal.testAllReferenceSets: 0.0019s
tests.unit.test_views.TestFrontend.testNoAuthentication: 0.0018s
tests.unit.test_views.TestFrontend.testRnaQuantificationsSearch: 0.0018s
tests.unit.test_views.TestFrontend.testGetRnaQuantification: 0.0018s
tests.unit.test_views.TestFrontend.testGetReadGroup: 0.0017s
tests.unit.test_views.TestFrontend.testSearchReadsMultipleReadGroupSetsSetMismatch: 0.0017s
tests.unit.test_backends.TestTopLevelObjectGenerator.testPageTokenNone: 0.0016s
tests.unit.test_file_handle_cache.TestFileHandleCache.testGetFileHandle: 0.0016s
tests.unit.test_views.TestFrontend.testGetCallSet: 0.0016s
tests.unit.test_views.TestFrontend.testDatasetsSearch: 0.0016s
tests.unit.test_exceptions.TestExceptionConsistency.testInstantiation: 0.0015s
tests.unit.test_views.TestFrontend.testRnaQuantificationSetsSearch: 0.0015s
tests.unit.test_views.TestFrontend.testGetExpressionLevel: 0.0014s
tests.unit.test_imports.TestImports.testNoCycles: 0.0013s
tests.unit.test_client.TestExhaustiveListingsHttp.testAllDatasets: 0.0012s
tests.unit.test_compound_ids.TestCompoundIds.testVariantParse: 0.0011s
tests.unit.test_exceptions.TestExceptionHandler.testUnknownExceptionBecomesServerError: 0.0010s
tests.unit.test_cli.TestServerArguments.testParseArguments: 0.0010s
tests.unit.test_cli.TestGa2VcfArguments.testParseArguments: 0.0010s
tests.unit.test_bio_metadata.TestIndividuals.testToProtocolElement: 0.0009s
tests.unit.test_bio_metadata.TestBioSamples.testToProtocolElement: 0.0008s
tests.unit.test_gff3_parser.TestGff3ParserOnSpecialCasesFile.testAllFeaturesContainAllRootFeatures: 0.0008s
tests.unit.test_gff3_parser.TestGff3ParserOnSpecialCasesFile.testInvalidFeatureNameKeyQueryFails: 0.0008s
tests.unit.test_cli.TestGa2SamArguments.testParseArguments: 0.0008s
tests.unit.test_datarepo.TestBadDatabase.testTextFile: 0.0007s
tests.unit.test_gff3_parser.TestGff3ParserOnSpecialCasesFile.testFileParsedHasSomeRootFeatures: 0.0007s
tests.unit.test_cli.TestOutputFormats.testJsonOutput: 0.0007s
tests.unit.test_gff3_parser.TestGff3ParserOnSpecialCasesFile.testAllChildrenFeaturesArePresentInSet: 0.0007s
tests.unit.test_gff3_parser.TestGff3ParserOnSpecialCasesFile.testRootFeaturesHaveNoParents: 0.0007s
tests.unit.test_gff3_parser.TestGff3ParserOnSpecialCasesFile.testSomeFeatureIsWellFormed: 0.0007s
tests.unit.test_cli.TestOutputFormats.testTextOutput: 0.0007s
tests.datadriven.test_references.testReferenceSets: 0.0006s
tests.unit.test_variants.TestAbstractVariantSet.testAddMultipleCallSets: 0.0006s
tests.unit.test_cli.TestOutputFormats.testListReferenceBasesFasta: 0.0006s
tests.unit.test_intervals.TestIntervalIterator.testEmptyInterval: 0.0006s
tests.unit.test_compound_ids.TestCompoundIds.testVariantSetMetadataParse: 0.0006s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchVariantAnnotations: 0.0006s
tests.unit.test_datarepo.TestBadDatabase.testDbFileWithoutTables: 0.0005s
tests.unit.test_compound_ids.TestCompoundIds.testExperimentParse: 0.0005s
tests.unit.test_compound_ids.TestCompoundIds.testVariantSetMetadata: 0.0005s
tests.unit.test_faultydata.TestVariantSetNoIndexedVcf.testInstantiation: 0.0005s
tests.unit.test_compound_ids.TestCompoundIds.testExpressionLevelParse: 0.0005s
tests.unit.test_compound_ids.TestCompoundIds.testReadGroupParse: 0.0005s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchIndividuals: 0.0005s
tests.unit.test_compound_ids.TestCompoundIds.testCallSetParse: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchBioSamples: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchReads: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchFeatures: 0.0004s
tests.unit.test_compound_ids.TestCompoundIds.testReadAlignmentParse: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetBioSample: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchExpressionLevels: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetReadGroupSets: 0.0004s
tests.unit.test_exceptions.TestExceptionConsistency.testCodeInvariants: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchReferenceSets: 0.0004s
tests.unit.test_references.TestAbstractReferenceSet.testAddMultipleReference: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchReadGroupSets: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchDatasets: 0.0004s
tests.unit.test_compound_ids.TestCompoundIds.testReadGroupSetParse: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchCallSets: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchRnaQuantificationSets: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchVariants: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchRnaQuantifications: 0.0004s
tests.datadriven.test_rna_quantification.testRnaQuantification: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchVariantSets: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchReferences: 0.0004s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetVariant: 0.0004s
tests.unit.test_compound_ids.TestCompoundIds.testRnaQuantificationParse: 0.0004s
tests.unit.test_compound_ids.TestCompoundIds.testVariantSetParse: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetCallSets: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetReference: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetReferenceSet: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchFeatureSets: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetRnaQuantification: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSearchVariantAnnotationSets: 0.0003s
tests.unit.test_compound_ids.TestCompoundIds.testExpressionLevel: 0.0003s
tests.unit.test_backends.TestAbstractBackend.testAddMultipleDatasets: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetExpressionLevel: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetDatasets: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetIndividual: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetReadGroup: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetVariantAnnotationSet: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testSetPageSize: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetRnaQuantificationSet: 0.0003s
tests.unit.test_client.TestSearchMethodsCallRunRequest.testGetVariantSet: 0.0003s
tests.unit.test_compound_ids.TestCompoundIds.testVariant: 0.0003s
tests.unit.test_compound_ids.TestCompoundIds.testFeatureSetParse: 0.0002s
tests.unit.test_file_handle_cache.TestFileHandleCache.testSetCacheMaxSize: 0.0002s
tests.datadriven.test_sequenceAnnotations.testFeatureSets: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testReferenceParse: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testExperiment: 0.0002s
tests.unit.test_exceptions.TestExceptionHandler.testCallSetNotInVariantSetException: 0.0002s
tests.unit.test_schemas.SearchResponseBuilderTest.testNextPageToken: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testReadAlignment: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testReferenceSetParse: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testDatasetParse: 0.0002s
tests.unit.test_variants.TestAbstractVariantSet.testAddOneCallSet: 0.0002s
tests.unit.test_oidc.TestFrontendOidc.testMimetype: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testRnaQuantification: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testCallSet: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testBadParse: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testURLUnsafe: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testReadGroup: 0.0002s
tests.unit.test_datarepo.TestBadDatabaseNoSetup.testDirectory: 0.0002s
tests.unit.test_compound_ids.TestCompoundIds.testVariantSet: 0.0002s
tests.unit.test_backends.TestTopLevelObjectGenerator.testPageToken: 0.0002s
tests.unit.test_references.TestAbstractReference.testReferenceRangeError: 0.0002s
tests.unit.test_exceptions.TestExceptionHandler.testNotImplementedException: 0.0001s
tests.unit.test_variants.TestAbstractVariantSet.testCallSetMethods: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testAddMultipleReferenceSet: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testFeatureSet: 0.0001s
tests.unit.test_variants.TestAbstractVariantSet.testHashVariant: 0.0001s
tests.unit.test_variants.TestAbstractVariantSet.testGetVariantId: 0.0001s
tests.unit.test_variants.TestAbstractVariantSet.testVariantSetProtocolElement: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testAttrs: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testTopLevelIdsUnique: 0.0001s
tests.datadriven.test_asserts.TestAsserts.testAssertAlmostEqual: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testUnicode: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testEncodeRoundTrip: 0.0001s
tests.unit.test_references.TestAbstractReferenceSet.testReferenceNotFound: 0.0001s
tests.unit.test_references.TestAbstractReferenceSet.testReferenceNameNotFound: 0.0001s
tests.unit.test_sequence_annotations.TestAbstractFeatureSet.testGetFeatureIdFailsWithNullInput: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testReadGroupSet: 0.0001s
tests.unit.test_exceptions.TestExceptionHandler.testObjectNotFoundException: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testGetDatasetByIndexBadIndex: 0.0001s
tests.unit.test_references.TestAbstractReferenceSet.testAddOneReference: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testReference: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testAddOneDataset: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testAddOneReferenceSet: 0.0001s
tests.unit.test_utils.TestCaptureOutput.testCapture: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testGetReferenceSetByIndexBadIndex: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testGetDatasetBadId: 0.0001s
tests.unit.test_reads.TestSamCigar.testSoftClipping: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testSplit: 0.0001s
tests.unit.test_reads.TestSamCigar.testSkipped: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testReferenceSet: 0.0001s
tests.unit.test_backends.TestPrivateBackendMethods.testParseIntegerArgument: 0.0001s
tests.unit.test_repo_manager.TestGetNameFromPath.testFullPaths: 0.0001s
tests.datadriven.test_asserts.TestAsserts.testAssertFalse: 0.0001s
tests.unit.test_repo_manager.TestGetNameFromPath.testDirectoryName: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testInstantiate: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testGetDatasetBadName: 0.0001s
tests.unit.test_response_generators.TestVariantsIntervalIteratorClassMethods.testGetVariantStart: 0.0001s
tests.unit.test_response_generators.TestReadsIntervalIteratorClassMethods.testGetReadEnd: 0.0001s
tests.unit.test_datarepo.TestBadDatabaseNoSetup.testNonexistantFile: 0.0001s
tests.datadriven.test_asserts.TestAsserts.testAssertEqual: 0.0001s
tests.unit.test_utils.TestCaptureOutput.testArgs: 0.0001s
tests.unit.test_response_generators.TestReadsIntervalIteratorClassMethods.testGetReadStart: 0.0001s
tests.unit.test_sanitizer.TestPysamSanitizer.testSanitizeString: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testGetReferenceSetBadName: 0.0001s
tests.unit.test_reads.TestParseMalformedBamHeader.testSpaceSeparatedUnparsedFieldsParsed: 0.0001s
tests.unit.test_backends.TestAbstractBackend.testGetReferenceSetBadId: 0.0001s
tests.unit.test_compound_ids.TestCompoundIds.testDataset: 0.0001s
tests.unit.test_reads.TestSamCigar.testInsertion: 0.0001s
tests.unit.test_utils.TestTimeout.testTimeoutNoException: 0.0001s
tests.unit.test_repo_manager.TestGetNameFromPath.testUrls: 0.0001s
tests.unit.test_reads.TestParseMalformedBamHeader.testOriginalTypesUnmodified: 0.0001s
tests.datadriven.test_asserts.TestAsserts.testAssertGreater: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertLess: 0.0000s
tests.unit.test_reads.TestSamCigar.testSequenceMismatch: 0.0000s
tests.unit.test_repo_manager.TestGetNameFromPath.testLocalDirectory: 0.0000s
tests.unit.test_compound_ids.TestCompoundIds.testGetInvalidIdString: 0.0000s
tests.unit.test_reads.TestParseMalformedBamHeader.testCommandsWithSpacesNotParsed: 0.0000s
tests.unit.test_sanitizer.TestPysamSanitizer.testAssertInRange: 0.0000s
tests.unit.test_reads.TestSamCigar.testHardClipping: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertTrue: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertIn: 0.0000s
tests.unit.test_sanitizer.TestPysamSanitizer.testSanitizeVariantFileFetch: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertIsInstance: 0.0000s
tests.unit.test_sanitizer.TestPysamSanitizer.testSanitizeInt: 0.0000s
tests.unit.test_compound_ids.TestCompoundIds.testEncodeDecode: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertIsNotInstance: 0.0000s
tests.unit.test_backends.TestPrivateBackendMethods.testParsePageToken: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertRaises: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertNotAlmostEqual: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertIs: 0.0000s
tests.unit.test_reads.TestSamCigar.testSequenceMatch: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertIsNot: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertIsNone: 0.0000s
tests.unit.test_reads.TestSamCigar.testPadding: 0.0000s
tests.unit.test_reads.TestParseMalformedBamHeader.testGoodHeaderUnmodified: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertGreaterEqual: 0.0000s
tests.unit.test_repo_manager.TestGetNameFromPath.testError: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertIsNotNone: 0.0000s
tests.unit.test_response_generators.TestVariantsIntervalIteratorClassMethods.testGetVariantEnd: 0.0000s
tests.unit.test_sanitizer.TestPysamSanitizer.testAssertInt: 0.0000s
tests.unit.test_reads.TestSamFlags.testFailedQualityCheckFlag: 0.0000s
tests.unit.test_sanitizer.TestPysamSanitizer.testAssertValidRange: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertLessEqual: 0.0000s
tests.unit.test_reads.TestSamFlags.testComboFlag: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertNotIn: 0.0000s
tests.unit.test_compound_ids.TestCompoundIds.testJoin: 0.0000s
tests.unit.test_sanitizer.TestPysamSanitizer.testSanitizeAlignmentFileFetch: 0.0000s
tests.datadriven.test_asserts.TestAsserts.testAssertNotEqual: 0.0000s
tests.unit.test_intervals.TestIntervalsIntersect.testIntersecting: 0.0000s
tests.datadriven.test_ontologies.testReferenceSets: 0.0000s
tests.unit.test_reads.TestSamFlags.testDuplicateReadFlag: 0.0000s
tests.unit.test_utils.TestRepeat.testRepeat: 0.0000s
tests.unit.test_reads.TestSamCigar.testDeletion: 0.0000s
tests.unit.test_reads.TestSamFlags.testReverseStrandReadFlag: 0.0000s
tests.unit.test_reads.TestSamFlags.testProperPairReadFlag: 0.0000s
tests.unit.test_reads.TestSamFlags.testFirstPairFlag: 0.0000s
tests.unit.test_reads.TestSamFlags.testSecondPairFlag: 0.0000s
tests.unit.test_reads.TestSamFlags.testPairedReadFlag: 0.0000s
tests.unit.test_reads.TestSamFlags.testUnmappedReadFlag: 0.0000s
tests.unit.test_reads.TestSamFlags.testSecondaryAlignmentFlag: 0.0000s
tests.unit.test_reads.TestSamFlags.testReverseStrandMateFlag: 0.0000s
tests.unit.test_reads.TestSamCigar.testAlignmentMatch: 0.0000s
tests.unit.test_intervals.TestIntervalsIntersect.testNonIntersecting: 0.0000s
tests.unit.test_reads.TestSamFlags.testSupplementaryAlignmentFlag: 0.0000s
tests.unit.test_reads.TestSamFlags.testFlagNotSet: 0.0000s
tests.unit.test_reads.TestSamFlags.testUnmappedMateFlag: 0.0000s
tests.unit.test_intervals.TestIntervalsIntersect.testEmptyIntervals: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetReadGroup: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetReferenceSet: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetRnaQuantificationSet: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchCallSets: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetVariantAnnotationSet: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetCallSet: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetRnaQuantification: 0.0000s
tests.end_to_end.test_client_json.TestClientFasta.testListReferenceBases: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchBioSamples: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchReferenceSets: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetBioSamples: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetIndividuals: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchDatasets: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchReads: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchReadGroupSets: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchVariants: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetReadGroupSet: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetDataset: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchVariantAnnotations: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchReferences: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetReference: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchVariantAnnotationSets: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchVariantSets: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testSearchIndividuals: 0.0000s
tests.end_to_end.test_client_json.TestClientJson.testGetVariantSet: 0.0000s
----------------------------------------------------------------------
Ran 804 tests in 122.848s

OK (SKIP=12)
dcolligan commented 8 years ago

Strangely, I find that the tests run about twice as fast if run via nosetests directly rather than run_tests.py ...

dcolligan commented 8 years ago

It looks like the coverage is a lot of the slowness... on my machine, nosetests runs 5m46s with flags, 3m18s without them. Not a lot we can do about that on travis... closing...

dcolligan commented 8 years ago

(This is a way to run travis steps locally with bash instead of python, not like that's really helpful to anyone:)

eval $(grep -A 100 '^script:' .travis.yml | xargs | cut -c 11- - | sed -e 's/ - / \&\& /g')

david4096 commented 7 years ago

We should try to find a way to parallelize some of the tests. Not all of them are written to be reentrant, so we might just run each test directory on its own process before we figure out which tests are properly isolated.

A script to do that should still report exit codes the same way and should run only as slow as the slowest test suite (datadriven I think). This doesn't speed up the tests but will hopefully improve perceived performance and taken better advantage of available resources. Any other ideas? 8 minutes for a Travis run seems long to me.

The other thing we ought to do is move the client tests out to the client.

dcolligan commented 7 years ago

nose has plugins to do this http://nose.readthedocs.io/en/latest/doc_tests/test_multiprocess/multiprocess.html

However, I'm sure this will expose tons of bugs in our tests, as moving to parallelize tests usually does...

david4096 commented 7 years ago

Yeah I've tried it out once or twice. Hence why I think we could just split them across processes in the shell, though Travis instances are pretty light on resources.

Maybe the first step would be isolating any more tests we can to the client.

The tests that don't stand up a server should usually run in multiprocess. I experimented with the _multiprocess_can_split_ = True and _multiprocess_shared_ = True variables to try to control which tests were being run under multiprocess without success.