![]() In the graph below, you can see that average test suite run time dropped by over 33%-from over 3 minutes to about 2 minutes. This reduced the combined overhead of all the tests to a total of about 10 seconds. ![]() When we swapped the 'TRUNCATE's for 'DELETE's, the overhead of each test dropped to 1-3 milliseconds. (In practice, we run groups of tests in parallel, so the perceived total overhead was closer to about 1 minute total.) In other words, this added 1 to 5 minutes of overhead for the entire suite when run serially. The 'TRUNCATE's accounted for 20-200 milliseconds of overhead for each test. Before each test runs, no fewer than 32 tables are wiped clean with a `TRUNCATE`. We were surprised when we measured the overhead of clearing table data. The entire suite runs before every commit is merged, as well as after each commit lands in master. The codebase has constant activity, and changes are continuously committed by authors from multiple teams. ![]() The nearly 3,200 tests included are responsible for the correctness of one of our most critical codebases. Our team's productivity depends on the performance of this test suite, since they run very frequently and are an integral part of the deployment pipeline. In one of our largest test suites comprised of thousands of tests interacting with Postgres, we reduced the run time of the suite by over 33% by switching our table-clearing SQL to use 'DELETE' rather than 'TRUNCATE'. While a clean database state for every test is beneficial in this sense, the overhead of clearing data can be significant. This ensures that the tests remain robust and immune to side-effects of adjacent tests. A common pattern in automated testing is to clear the database state before every test runs.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |