Changelog
Adding GitHub repositories and references (branches, tags, pull requests) in TrustInSoft CI was not clear enough: the first reference was selected from the "Add project" page but the next references were automatically added under some conditions.
Now, the action of adding a GitHub repository is separated from the action of adding a repository's reference. A build can be run for a reference only if the reference has been added in TrustInSoft CI first.
The button to add a GitHub project has been moved from the navigation bar to the account's projects page:
In the same way, the button to add a reference of a project can be found in the project page:
All builds of a single GitHub repository were listed all together in the project's page. It also means that using the
Previous or Next buttons on a build's page (to go to the previous/next build of the same project) could lead to a build of another branch/tag. So comparing results for a same reference was not convenient.Now, the project's page contains the list of the project's references added in TrustInSoft CI (with the result of their latest build). Then a new page have been added for a project's reference which lists all the builds related to the said reference:
The tinycrypt project's page lists only the references of this project.
Builds of a reference are no longer mixed with builds of other references.
This change also impacts the URL addresses of builds, which means if you bookmarked or shared an URL leading to a build or analysis, they are no longer valid. We try to minimize breaking changes but this one cannot be avoided, sorry for the inconvenience.
⚠
But we still kept the support for
tis.config files, so we will not have to change anything for projects you already set-up in TrustInSoft CI.Also, the syntax of configuration files is unchanged. We decided to rename this file to use the
.json extension (that makes IDE/editors happy for syntax highlighting), and to allow the usage of a directory to gather any files related to TrustInSoft CI in a single place (see also the next new feature!).Do you need to perform some actions before running the analyses described in
.trustinsoft/config.json?Create a new
.trustinsoft/prepare.shscript in your GitHub repository and write in bash all the actions you need to do, such as: - Generating some missing source files or artifacts useful for analyses
- Cloning external libraries to make their source code available for analyses
- Generating the
.trustinsoft/config.json - And so on...
This
.trustinsoft/prepare.sh script is run inside the .trustinsoft/ directory only once per build before reading the .trustinsoft/config.json . Read more about the prepare.sh file in the documentation.Do you want to quickly set-up a repository on TrustInSoft CI without committing files? Did you already create a new branch to commit TrustInSoft CI configuration files to avoid committing them in the master branch?
It is now possible to write TrustInSoft CI configuration files (
.trustinsoft/config.json and .trustinsoft/prepare.sh files) directly in the Project settings page:The Build configuration settings is global to the project and is used for all references of this project, except if a
.trustinsoft directory exists among the committed files for the analyzed reference (in this case, committed files are taken into account and the global configuration is ignored).Hence, it is recommended to use the Global configuration to quickly set-up a project in TrustInSoft CI, and to use committed files when the configuration differs from a branch to another.
Also, each build clearly indicates whether it is using the Global configuration of the project or the committed files for the performed analyses:
With the new improved workflow to set-up a project, new settings have been added in the Project settings page to let you configure:
- if references should automatically be added in TrustInSoft CI, to avoid clicking on
Add referenceto many times, especially when Pull Requests are opened - when builds are triggered, to avoid to run a build when a commit is pushed on GitHub (if you prefer to keep the full control by running build manually)
Inspecting an analysis with TrustInSoft CI Analyzer may take time (because the analysis needs to be re-run again). So the analysis summaries have been improved to reduce the necessity of inspecting the analysis with TrustInSoft CI Analyzer (but it will still be necessary on some cases).
For instance, a lot more information are available when an undefined behavior has been detected:
All
Inspect buttons have been moved into the Summary tabs when an analysis is expanded.With all the new features, we decided to rewrite the documentation to make the set-up of a project as clear as possible.
For instance, if you want to learn more about the Configuration files, how and where to write them, feel free to read the new Configuration files section!
The TrustInSoft CI homepage (visible when you are not logged-in) has been changed to explain more in details (but without being too technical) about how TrustInSoft CI can help you and how it works.
If you are interested in our technology and want to do more formal analyses, you can find the list of our other services and products on this page.
This page should help to convince your boss or friends to do more than just unit testing!
- The life chat has been removed. The dust was stacking too much in this widget.
- A glossary has been added in the documentation. Because the words used in TrustInSoft CI are not obvious for everyone.
- Minor bug fixes and improvement. As usual!
Previously, you had to use your browser history or bookmark to go back to a visited project that you do not own. And if you have a lot of projects, you may get lost in the project lists.
This is the past. Now, a new Dashboard is available to focus on your recent activity, including your latest updated projects and the latest projects you visited.
Some shortcut links are also available at the top of the Dashboard. Let us know which action you use the most to include it as a shortcut in this page in the future.
Do you want to find a particular project or user in TrustInSoft CI?
The new Search input is available at the top-right of the page to allow you to find and browse any project or user using TrustInSoft CI.
As a bonus, if you click in this input without writing anything, the latest project you visited will be shown. It allows to switch between projects more efficiently without going through the Dashboard case!
Never two without three.
You are missing inspiration? You want to look at how other users are using TrustInSoft CI? Do you want to know more about the fine details of the C standards? Or just browsing the Internet during the coffee break?
The new Explore page is for you! This page will display various things such as:
- the most popular projects and users of TrustInSoft CI (but the popularity algorithm is kept secret!)
- the latest blog posts of our C/C++ experts
- some hints about our other services if TrustInSoft CI do not cover all your needs
A new settings page is available for each project (only for projects you owned). Settings related to this project will be shown on this page.
Also, you can now allow builds to be automatically canceled if a newer one is in the queue for the same reference (branch/tag/Pull Request).
With all these new features, the navigation bar has been entirely re-designed, to keep a quick access to any useful feature.
We also care about small devices. So, we improved the responsive design, allowing results to be easily read from smartphones and tablets too.
The contrast of some color has also been increased in both the application and the email notifications. It should now be easier to spot problem at a glance.
The "Explore" button to inspect in detail the analyses result of a test has been renamed into "Inspect" (to not be mistaken with the new "Explore" feature).
Also, the "Inspect" feature has also been disabled for tests using the
"no-result": true field. With this field, nothing relevant can be inspected within the Analyzer view.And of course, a lot of various invisible improvements and bug fixes have been done to make your visit on TrustInSoft CI as comfortable as possible.
Previously, guest users had no choice but to create a TrustInSoft CI account to browse analysis results with the Analyzer. As TrustInSoft CI user, this made it impractical when you wanted to question someone, who'd never used TrustInSoft CI, about an Undefined Behavior.
Now, you can click on an Explore button, copy-paste the Analyzer URL in the address bar and share it with anyone, who can then freely look at the Undefined Behavior, without ever having to create an account:
From now on, a new analysis will be triggered every time a pull request is created / updated or a git tag is pushed to your remote repository. Also learn if any given analysis refers to a branch update, a pull request or a tag by reading the REFERENCE columns which replaces the BRANCH column.
Some users have asked us to share a detailed description of each supported target architecture so they can pick the right subset of architectures to run their analyses on and therefore save analysis time.
☑
It's done! The csv file available here contains, for each supported architecture, information such as
char size, short size, int size, etc. Use one or both of these two fields to let TrustInSoft CI know how to preprocess the source files in your project prior to the analysis.
You can provide either the preprocessing options that are relevant to your project and the analysis (such as
-I, -D or -U ) or the whole compilation line. TrustInSoft CI will keep only the relevant options.Use
cpp-extra-args and cxx-cpp-extra-args for respectively C and C++ source files. If your project contains source files in both languages, include both fields in the tis.config file.These fields replace and deprecate the
compilation_cmdfield. Please replace it withcpp-extra-argsand/orcxx-cpp-extra-argsin all your tis.config file!Previously, after you clicked on the Explore button, a new tab opened to load the Analyzer. When by chance the opening of this tab wasn't already blocked by your browser, this behavior was painful, since you had to go back and forth between the main TrustInSoft CI tab and this Analyzer tab.
You can now share Analyzer URLs with other logged-in users!
☑
And in an upcoming release, share them with logged-out users too! For example, it can be quite handy when you want to question someone about an Undefined Behavior found. Click on an Explore button, copy-paste the Analyzer URL in the address bar and share it with others:
Some of your tests are taking too long or are running into an out-of-memory? Feel free to add this line
"no-results": true to your tis.config file. The tests will run faster and consume less memory.Warning: the values of the variables will no longer be available in the Analyzer view.
New field
☑
cxx-std in the tis.config file. Use this option to specify the C++ standard to follow. Default is c++11.New column "EXIT STATUS" in the Tests table.
☑
By providing details on how your test terminated, this new column enables you to quickly spot the tests that prematurely exited or aborted. Hover your mouse over the Question mark icon to learn about the possible values.
When
"no-results"is set in the tis.config file, values in this column may be missing in some cases. We are working on it.Other small improvements and fixes
☑
Start receiving email notifications every time a verification is done! Several email trigger options are available. Learn more
☑
The field
☑
raw_options is now deprecated: please remove all of its occurrences from your tis.config files. Here is an example of tis.config file before and after the requested change:tis.config before change
tis.config after change
{
...
"raw_options": {"-val-timeout": "%d"}
...
}
{
...
"val-timeout": %d
...
}
Other small improvements and fixes
☑
New test status: Invalid property.
☑
You may for example use this new status to verify the result of your tests. Annotate your code with an ACSL property. If TrustInSoft CI fails to verify such property, the test will fail with this status.
In the below example, TrustInSoft CI will check if the
vector_test function returns 0 and it will return the Invalid property status otherwise://@ ensures \result == 0;
int vector_test(void (*f)(const vector[], vector*),
const char *name, size_t nb_inputs,
size_t nb_vectors, u8 **vectors, size_t *sizes)
Various small improvements and fixes especially on test statuses, filter and pagination
☑
See 20 tests at a time with the new pagination component
☑
Use the new test status filter in order, for example, to focus on the tests with an undefined behavior
☑
You can now override the default 15-minute timeout value per test by adding the new option
☑
val-timeout to your tis.config file as follows:"val-timeout": <v>, where <v> is the new timeout value per test in seconds between 60 and 10800 (3 hours) Other small improvements and bug fixes
☑
You can now view up to 1000 tests or verifications on the same page versus 50 before
☑
The memory allocated to verify each test has been doubled from 1.7GB to 3.5GB
☑
For a given verification, you may now see, next to the Verification Result field, either a Warning icon for you to take action (add or fix your tis.config file) or a Cross icon to inform you that the Verification Result is either missing or incomplete (Git error or canceled verification)
☑
Other small improvements and bug fixes
☑
More granular Test Status: when your test has encountered an error, you do not need to click the test anymore to know the exact error type, just read the more granular Test Status field
☑
More detailed Verification Status: the Verification Status field now displays the detailed tests results instead of a consolidated result. As a result, the Tests Summary column has been removed (as redundant)
☑
New Delete button on the Verification history/project page: feel free to delete select verifications within your project history to share a clean TrustInSoft CI verification history with others
☑
The analysis of programs, that contain floating-point values, is now more precise and can handle infinities and NaNs
☑
Other small improvements and bug fixes
☑
Easy configuration: error messages related to missing or invalid tis.config files are more precise
☑
The undefined behavior `Overflow` becomes `Signed overflow`
☑
TrustInSoft CI Analyzer re-loads much faster: from 50s down to 10s for a 5s analysis (except the first time the Analyzer is loaded)
☑
Other small improvements and bug fixes
☑
Bye bye Firebase and the reliance on third-party cookies: authenticate to TrustInSoft CI directly via GitHub
☑
Easy configuration: on the Add project page, click on the Copy button to copy-paste a sample tis.config file and add it to your project
☑
Easy sharing: copy-paste test URLs and share with others by clicking on the Hyperlink button
☑
After launch, TrustInSoft CI Analyzer now automatically points to the code statement where the Undefined behavior was found
☑
If your project or GitHub organization changes name, TrustInSoft CI automatically updates the database so the change is transparent to you
☑
Other small improvements and bug fixes
☑
Easy configuration: there is a new Config tab that lets you view each of your test configuration without having to switch to github.com
☑
Other small improvements and bug fixes
☑
Last modified 1yr ago