1 Topic by dbosart

  • dbosart
  • Member
  • Offline
  • Registered: 2009-10-22
  • Posts: 20

Topic: Changing a configuration after test results are posted

We had a situation where the configuration (in this case, the OS) denoted in the test run differed from the target (32 bit vs. 64bit) but the test results were posted and were valid test results.  Unfortunately, when we tried to correct the configuration, the test results were reset to "Untested".

Obviously we made an error; but, equally obvious if the run contains a lot of tests, correcting the results is a time intensive chore.

I'm not sure what the tool could do to mitigate this situation; but, I'd be interested in any ideas.

2 Reply by tgurock

  • tgurock
  • Administrator
  • Offline
  • Registered: 2009-07-09
  • Posts: 628

Re: Changing a configuration after test results are posted

Hello,

Some background info first: when you change the configurations of a suite in a test plan, TestRail calculates a diff with the test runs that are no longer wanted, the test runs that must be updated and those that must be added (this diff is shown when you hit the save button).

So, when you change the config from 32-bit to 64-bit, TestRail deletes the entire test run with the 32-bit config and adds a new test run for 64-bit. That's the reason why your tests were 'untested' after you edited your test plan (because they are essentially newly created tests). We are looking into ways to improve the flexibility of configurations, although I'm not sure if there's a good way to handle these scenarios. There are a few other edge cases which aren't perfect yet and I'm sure we will enhance the configuration handling until we ship the 1.0. Thanks a lot!

Regards,
Tobias