 |
Matrix Science Mascot Parser toolkit
|
|
|
Matrix Science Mascot Parser toolkit
|
|
The class ms_mascotresfile has been renamed to ms_mascotresfile_dat. The class now inherits from ms_mascotresfilebase, as part of implementing support for parsing Mascot Search Results (MSR) files. The methods and members of ms_mascotresfile_dat are otherwise identical to ms_mascotresfile in Parser 2.8.
Because the API remains the same, client-side code is straightforward to update.
perl -c -w your_script.pl. Perl should point out remaining instances where "ms_mascotresfile" is still used. Note: The Perl compiler will warn you about cases where ms_mascotresfile is used as a bare identifier, but it will not warn you about cases where you are using class variables like enums from ms_mascotresfile, such as $ms_mascotresfile::RESFILE_NOFLAG.
Note: Although most Python code is compiled to intermediate bytecode, the compiler does not provide full static analysis or type checking. We recommend exercising every code path in your program if you don't already have unit tests in place.
Mascot Server 3.0 defaults to writing search results in Mascot Search Results (MSR) format, which is an SQLite file. The new class ms_mascotresfile_msr provides full support for reading MSR files. ms_mascotresfile_dat and ms_mascotresfile_msr inherit from the new base class ms_mascotresfilebase.
All Parser results file classes, such as ms_peptidesummary and ms_inputquery, have been updated to support reading from ms_mascotresfile_msr. The relevant constructors and methods now take an ms_mascotresfilebase reference as input, instead of ms_mascotresfile_dat. All the API remains otherwise the same, which means supporting both .dat and MSR formats can be done with a few small changes.
| Old method call | Intent | Replacement |
|---|---|---|
| ms_mascotresfile::willCreateCache(filepath, ms_mascotresfile::RESFILE_USE_CACHE, cacheDirectory) | Will a cache file be created? | ms_mascotresfilebase::willCreateCache(filepath, ms_mascotresfilebase::RESFILE_USE_CACHE, cacheDirectory) |
| ms_mascotresfile::doesSectionExist(SEC_DECOYSUMMARY) ms_mascotresfile::doesSectionExist(SEC_ERRTOLDECOYSUMMARY) | Is this a target-decoy search? | ms_searchparams::getDECOY() > 0 |
| ms_mascotresfile::doesSectionExist(SEC_ERRTOLPEPTIDES) | Sometimes used to test whether this is an ET search | ms_mascotresfilebase::isErrorTolerant() |
| not ms_mascotresfile::doesSectionExist(SEC_ERRTOLSUMMARY) | Often used in conjunction with ms_mascotresfile::isErrorTolerant() to detect a manual ET search (aka old-style ET search) | ms_mascotresfilebase::isErrorTolerant() && not(ms_searchparams::getErrTolParentFilename() == "") (not enough to test ms_searchparams::getERRORTOLERANT()) |
| ms_mascotresfile::doesSectionExist(SEC_SPECTRAL_LIBRARY) or ms_mascotresfile::doesSectionExist(SEC_LIBRARYSUMMARY) or ms_mascotresfile::doesSectionExist(SEC_LIBRARYPEPTIDES) | Often used to detect a spectral library search | ms_mascotresfilebase::anySpectraLibraryMatches() |
| ms_mascotresfile::doesSectionExist(SEC_CROSSLINK_SUMMARY) or ms_mascotresfile::doesSectionExist(SEC_CROSSLINK_PEPTIDES) | Often used to detect a crosslink search | ms_mascotresfilebase::anyCrosslinkedMatches() |
| ms_mascotresfile::getSectionValueStr(SEC_HEADER) ms_mascotresfile::enumerateSectionKeys(SEC_HEADER) | Read values from the results file header | Either use ms_searchparams methods (preferred) or the lower level ms_mascotresfilebase::getHeaderKeyValues() |
| ms_mascotresfile::getSectionValueStr(SEC_PARAMETERS) ms_mascotresfile::enumerateSectionKeys(SEC_PARAMETERS) | Read search parameters directly | Either use ms_searchparams methods (preferred) or the lower level ms_mascotresfilebase::getSearchParametersKeyValues() |
| ms_mascotresfile::getSectionValueStr(SEC_MASSES) ms_mascotresfile::enumerateSectionKeys(SEC_MASSES) | Read fixed mod and variable mod masses directly | Either use ms_searchparams methods (preferred) or the lower level ms_mascotresfilebase::getMassesKeyValues() |
| ms_mascotresfile::getSectionValueStr(SEC_PEPTIDES) or other peptides section | Often used in places where new type of data has been added to the results file, but before new Parser API had been introduced. | Replace with a suitable higher-level API method, usually ms_peptide or ms_peptidesummary. |
| ms_mascotresfile::getSectionValueStr(SEC_PEPTIDES, "q1_p1_na_diff") | Concrete example of the above | New methods introduced in 3.0, ms_mascotresults::getErrTolOriginalNaSeq(), ms_mascotresults::getErrTolModifiedNaSeq() |
| ms_mascotresfile::getQuerySectionValueStr() ms_mascotresfile::enumerateQuerySectionKeys() | Read query-level attributes directly | Create ms_inputquery and use its methods |
| ms_mascotresfile::anyPeptideSummaryMatches(SEC_*) | Are there any peptide matches of the specified type? | Use one of: ms_mascotresfilebase::anyFastaMatches(), ms_mascotresfilebase::anySpectraLibraryMatches(), ms_mascotresfilebase::anyErrorTolerantMatches(), ms_mascotresfilebase::anyCrosslinkedMatches() |
Support for Visual Studio 2012 and 2013 has now been dropped. The Parser libraries have been compiled with VS2015 and support the VS2015-VS2022 runtime.
The MinGW static and shared libraries are now compiled with GCC 7.1.0. If you use the static Parser library with MinGW, please use GCC 7.1.0 or later.
Parser on Linux is now compiled with GCC 5.5.0. If you use the static Parser library, please use GCC 5.5 or later. The shared library may work with earlier compiler versions.
Support for 32-bit Windows has been dropped.
Static libraries for Intel Parallel Studio XE 2016 and later are no longer shipped with Parser. Please use the GCC 4.8.5 compiled libraries, which are binary compatible with the Intel compiler.
Mascot Server 2.8 added target-decoy support for error tolerant searches. The results file has new sections et_decoy_summary and et_decoy_peptides, which are analogous to et_summary/decoy_summary and et_peptides/decoy_peptides, respectively. Parser classes automatically read the new sections if they are present.
If the error tolerant search is from Server 2.8 or later, error tolerant matches now have expect values denoting statistical significance. See Error tolerant searches for more information.
If the error tolerant search is from Server 2.7 or earlier, expect values are not available. Previous Parser versions reported a non-zero, numerically incorrect value. It's best to check the return value of ms_mascotresults::getPeptideExpectationValue(): if it's -1, expect value is not available.
If the search is error tolerant and from Server 2.8 or later, Parser applies separate significance thresholds (minProbability) to first pass and second pass (ET) matches. These are available through ms_mascotresults::getProbabilityThreshold() and ms_mascotresults::getErrTolProbabilityThreshold(). The active thresholds may differ from the minProbability parameter passed to the constructor if the TargetFDR parameter is specified.
Because there are two thresholds, IgnoreIonsScoreBelow is restricted to only two values, making it into a binary choice.
In the latter case, Parser uses either the first or the second pass threshold to suppress non-significant matches.
If the search is not error tolerant, IgnoreIonsScoreBelow operates as an expect value or score filter like in previous versions. The value -1 ("automatic") is available in both ET and non-ET searches.
Parser now has complete support for MS1 quantitation using Mascot Distiller project files. See Quantitation analysis for Mascot Server and Distiller.
Support for Visual Studio 2005, 2008 and 2010 has been dropped.
Support for GCC 4.1 and Intel Parallel Studio XE 2013 has been dropped. Static libraries are now compiled with GCC 4.8.5 and Intel Parallel Studio XE 2016. The shared libraries remain compatible with GCC versions older than 4.8.
Support for 32-bit Linux has been dropped.
Mascot 2.7 introduced support for intact crosslinking and cleavable crosslinking. The results file contains a new section for intact crosslinked matches.
The default mode in Parser 2.7 and later is to mimic 2.6. That is, if you use the default flags, for example from get_ms_mascotresults_params(), then the file is opened in "linear-only" mode. If you want to see the intact crosslinked peptide matches, specify MSPEPSUM_CROSSLINK_INTEGRATED as the special mode flag to the ms_peptidesummary constructor.
Opening the files in Parser 2.6 and earlier is possible. Older versions will simply ignore intact crosslinked matches. If there are any matches with monolinks, however, client code will report the intact linker mass as the modification delta.
See Crosslinked search results for more information.
The matrix_science::ms_spectral_lib_file::FILE_FORMAT enum has been replaced by the matrix_science::ms_spectral_lib::FILE_FORMAT enum.
Mascot 2.6 introduced integrated spectral library searches, where the peak lists are searched simultaneously against FASTA database(s) and NIST-format spectral libraries. The results file contains additional sections for spectral library metadata and matches to library entries. There are no changes to the non-library sections of the file.
Opening the files in Parser 2.5 and earlier requires no changes to your code; the file works as if it contained no spectral library data.
The default mode in Parser 2.6 and later is to mimic 2.5. That is, if you use the default flags, for example from get_ms_mascotresults_params(), then the file is opened in "Mascot-only" mode. If you want to see the library matches, specify either MSPEPSUM_SL_INTEGRATED or MSPEPSUM_SL_ONLY as the special mode flag to the ms_peptidesummary constructor.
See Spectral libraries for more information.
In Mascot 2.6, Modification Statistics are based on modifications observed in rank 1 significant matches assigned to a protein hit, whereas they used to be based on matches which could be in the unassigned list.
This change applies to methods matrix_science::ms_mascotresults::getNumDiscoveredFixedMods(), matrix_science::ms_mascotresults::getNumDiscoveredVariableMods(), matrix_science::ms_mascotresults::getNumDiscoveredLocalMods(), matrix_science::ms_mascotresults::getDiscoveredLocalModNames(), matrix_science::ms_mascotresults::getNumDiscoveredErrTolMods(), matrix_science::ms_mascotresults::getDiscoveredErrTolModNames(), matrix_science::ms_mascotresults::getDiscoveredErrTolModDeltas(), matrix_science::ms_mascotresults::getNumDiscoveredLibraryMods(), matrix_science::ms_mascotresults::getNumDiscoveredNonSpecCleavage() .
In Mascot Parser, functions and classes that access Unimod or Quantitation XML configuration files require the schema files in order to validate the XML content. The required schema files are supplied with Mascot Parser in the config directory. The Mascot result files, (accessed using the ms_mascotresfile class), also contain a copy of the relevant part(s) of the XML configuration files that were used during the search. In Mascot 2.4 and earlier, some ms_mascotresfile member functions, or functions that pass a ms_mascotresfile object also require the relevant schema file. To avoid having to specify the schema file in multiple places, a directory can now be passed to the constructor to specify the location of the schema files. Additionally, a new function: setXMLschemaFilePath() may be called for finer grained access.
To ensure backward compatibility, it was necessary to add an additional parameter (with a default value) to ms_mascotresfile::getUnimod. If you are using this function, then this version of Mascot Parser is backward compatible with your existing code. However to simplify your code, if you have passed the relevant directory to the ms_mascotresfile constructor, then you may wish to remove the call to ms_umod_configfile::setSchemaFileName and then pass 'true' as useSchemaFromResfile when calling ms_mascotresfile::getUnimod.
It is strongly recommended that applications using Mascot Parser now:
For consistency and simplicity, ms_mascotresfile now inherits from ms_errors.
The enumerated definitions for errors declared in error numbers are now deprecated. The error numbers defined at ms_errs::err should be used instead for new applications. The numbers for the two lists are the same and have not been changed in any version of Parser, thus ensuring backward compatibility.
The following functions
just call the relevant ms_errs functions.
In Mascot Parser 2.1 and later, it is possible to download and parse configuration files from a remote Mascot Server using http. When accessing these files remotely, for example, using ms_enzymefile it is necessary to supply an ms_connection_settings object. In versions prior to 2.5, when the Mascot Server has authentication enabled, an http username and password needed to be supplied in the url for the ms_enzymefileobject. In Parser 2.5 and later, it is more convenient to supply these values using: ms_connection_settings::setHttpUsername and ms_connection_settings::setHttpPassword
Mascot Server 2.5 now allows locally defined modifications to have restricted access based on user groups. If security is enabled then Mascot Parser 2.5 will check these restrictions and remove modifications from the ms_modfile and ms_modfile object instances. During construction, passing a valid ms_connection_settings object with a session_id into these instances will apply the restrictions for that user. If no ms_connection_settings object is used then the default session will be employed. This behaviour will work for both fat and thin client applications.
Support for Visual Studio 6 and VS2003 has been dropped.
Support for VS2010 and VS2012 has been added.
Windows SDK 2003 SP1 required for Visual Studio 2005
For all versions of Mascot, it has been possible to search using multiple variable modifications for the same residue. In this case, Mascot searches for one modification or the other on each particular site. It never searches for both modifications being applied to the same site.
In Mascot Server 2.4 and later, it is possible to search for two modifications being applied to the same site. This is most useful with chemical labels used for quantitation, so it is currently only implemented when using a Quantitation method with exclusive modifications. For example, with a SILAC method, where the label is Label:13C(6)15N(2), it is also possible to specify Acetyl (K) as a variable modification. When Mascot is searching for the 'heavy' SILAC component it will look for:
The function matrix_science::ms_peptide::getVarModsStr() returns a single string from the q1_p1= line in the results file as before. A new function matrix_science::ms_peptide::getSummedModsStr() returns a new string of the same format.
For example, consider the case where the SILAC label is specified as an exclusive modification, and Methyl (KR) and Acetyl (K) are both specified as variable modifications.
In the case where K1 has the SILAC label + Methylation and K6 has the SILAC label + Acetyl, the results file would contain:
delta1=42.010565,Acetyl (K) NeutralLoss1=0.000000 . . . delta3=14.015650,Methyl (KR) NeutralLoss3=0.000000 . . . delta6=8.014199,Label:13C(6)15N(2) (K) NeutralLoss6=0.000000 q3_p1=2,1057.657852,0.000561,8,KQLATKAAR,17,06000060000, . . q3_p1_summed_mods=03000010000
matrix_science::ms_peptide::getVarModsStr() would return "06000060000" and matrix_science::ms_peptide::getSummedModsStr() would return "03000010000"
This new functionality is only available when MULTI_SITE_MODS is defined in the search input and this is not enabled by default. See matrix_science::ms_searchparams::getMULTI_SITE_MODS
When using matrix_science::ms_aahelper, use the new overloaded version of createPeptide with the summedMods and summedModsNL parameters. The varModVector passed to matrix_science::ms_aahelper::setAvailableModifications should obviously include the summed mods and the variable mods.
In Mascot 2.3 and later, ms-ms searches can also be run through Percolator, and the Percolator scores can then be shown in the Mascot reports. See Using Percolator scores for details.
There is now support for searching multiple fasta files. A search against a single fasta file will create a results file similar to that in Mascot 2.2. However, when searching multiple fasta files, remember that the same accession may exist in different fasta databases. The following are additional/changed functions that may be useful:
New protein grouping. See Using MSRES_CLUSTER_PROTEINS. Software needs to specify the new flag and call matrix_science::ms_mascotresults::getNextFamilyProtein as described in Using MSRES_CLUSTER_PROTEINS
It is now possible (but not required) to cache results. This is fully described in Caching Mascot Results
An updated schema, version 2.1 (quantitation_2.xsd) is supplied with Mascot Parser 2.3. Version 1.0 was supplied with Mascot Parser 2.x and version 2.0 was supplied with Mascot Distiller 2.3. To support files (for example results files) with quantation_1 and quantation_2 formats, both schema must be made made available.
The following calls worked with Mascot Parser 2.1 and 2.2, but will fail with Mascot Parser 2.3 and Quantiation 2:
$objResFile = new msparser::ms_mascotresfile("F001234.dat");
$objQuantFile = new msparser::ms_quant_configfile();
if ($objResFile->getQuantitation($objQuantFile)) {
# abort on fatal error. Note that isValid() remains true if there are only warnings
unless ($objQuantFile->isValid) {
. . .
}The schema filenames now need to be set before the getQuantitation() call as described in ms_mascotresfile::getQuantitation
Please contact Matrix Science support if you experience any issues upgrading from Parser 2.1 or earlier.
1.9.6