ForewordIntroductionScopeConformanceNormative referenceDefinitionsAbbreviations and symbolsAbbreviationsSymbolsThe measurementConfiguration requirementsUser emulationRandom user behaviourRemote terminal emulatorWorkload parameter setParameter set for proving the accuracy of the user emulationThe measurement procedureThe time phases of the measurement procedureWriting a measurement logfileWriting a computation result fileProof of validity of the measurementProof of the CBSS's computational correctnessProof of the remote terminal emulator's accuracyProof of the measurement result's statistical significanceCalculation of the performance values of the SUTMean execution timeThroughputTimely throughputBasic data for ratingUser requirementsThe reference environment for rating software efficiencyReference environment for assessing application software efficiencyReference environment for assessing system software efficiencyRating the performance valuesComputing the performance reference valuesMean execution time reference valuesThroughput reference valuesComputing the performance rating valuesThe mean execution time rating valuesThroughput rating valuesThe timeliness rating valuesRating the overall performance of the SUTAssessment of performanceThe steps of assessment processWeak reference environmentInput requirementsThe SUT descriptionSpecification of the hardware architecture and configurationSpecification of the system software configurationThe application programsAdditional software required for the measurement runThe stored dataAdditional information for proofThe workload parameter setThe activity typesActivity input variationThe task types with timeliness function and task modeThe chain types and their frequenciesPreparation times' mean values and their standard deviationsInput for measurement validationCorrect computation resultsVariation of input data and its resulting outputCriteria for precision of working of the RTECriteria for statistical validity of resultsThe measurementThe measurement procedureIndividual rating intervalOutput from measurement procedureMeasurement logfileComputation result fileValidation of measurementsValidation of the computational correctness of the SUTValidation of the accuracy of the RTEValidity test by checking the relative chain frequenciesValidity test by checking the preparation timesValidation of the statistical significance of the measured mean execution timeCalculation of the performance values of the SUTMean execution timeThroughputTimely throughputRating the measured performance values of the SUTSpecification of rating levelComputing performance reference valuesMean execution time reference valuesThroughput reference valuesComputing rating valuesComputing mean execution time rating valuesComputing throughput rating valuesComputing timeliness rating valuesRatingMean execution time ratingThroughput ratingTimeliness ratingOverall ratingSpecification of the RTE's basic functions (normative)Additional calculation formulas (normative)Formula of mean execution time reference value T(j)Formula of throughput reference value B(j)Algorithm for computing the timely throughput E(j)Formula for computing the indicators of the RTE's precision of workFormula DIFFFormula DIFFFormula DIFFFormulas for mean value and standard deviationThe sequential testFormat of the workload description
(normative)Workload parameter setBasic parameter valuesActivity types valuesTask types valuesTimeliness functions valuesChain types valuesRelative chain frequenciesPreparation time mean valuesStandard deviation values of preparation timesApplication programsOperating system command proceduresComputational resultsResults of activity type 'fixed input'Results of activity type 'input variation'Stored dataStatistical parameters and accuracy parametersCriteria for sufficient precise working of the RTECriteria for the statistical significance of the measurementFormat of the logfile (normative)Utility programs (informative)Program TREFREProgram BETAREProgram BEProgram DIFFProgram MANDSProgram SEQTESTExamples of workloads (informative)Workload SIMPLOAD1Workload SIMPLOAD2Workload SIMPLOAD3Workload COMPCENTER1Workload COMPCENTER2Workload COMPCENTER3