Regression testing is re-running tests to check that previously developed and tested code still performs after a change in the code. In FPGA development we often write many testcases to check the functional correctness of our design, and testcases can be run many times during the development process.
We know that proper verification can be time-consuming and that often more than half the project development time is spent on verification including writing testcases, structuring, debugging, and keeping track of testcase results and requirements. Setting up the simulation of the complete test suite with proper testcase traceability can, depending on the size and complexity of the design under test, be tedious and time-consuming.
With regression testing we test frequently, and we often write tests for small modules - almost like unit tests, test large modules and top-level designs. Complex designs can have numerous testcases, thus automating the verification flow is a big advantage and sometimes unavoidable.
HDLRegression is a test automation regression system for structuring the test flow and simulating testcases, all controlled from a single test script written in Python 3. With HDLRegression the verification engineer can easily setup the complete verification flow and quickly start simulating and check the functional correctness of the design. Setting up a test script can be done in minutes, and with the power of the Python language we can create advanced and effective test scripts to set up testbench generics and control testcases.
Some of the benefits of using HDLRegression to control the test suite:
• Easy to setup with new and existing projects through provided template files.
• Simple simulation control from command line
• Interactive recompilation and testcase control from Questa and Modelsim GUI - and easy to extend with more simulators.
• Continous testcase simulation status.
• Verbosity control
• Ensure all testcases are run
• Use with any verification framework
• Veriolog and VHDL support
• Automatic compilation and re-running of testcases
• Run on CI servers