First, give conclusions that can be used for decision-making
Create independent testing videos and label real events by people. The rate of underreporting is the proportion of real events not systematically detected, the rate of misreporting is the percentage of system alarms, but it is important to specify the time window, area, trajectory consolidation and confidence threshold. High-risk underreporting and ordinary misreporting cannot be covered by an overall accuracy rate.
What conditions need to be identified before judgement is made?
The same question may have different answers under different business, data and project phases. It is suggested that the following conditions be checked and that the common findings on the web be incorporated into their own projects.
Suggested order of advance
First, we'll be clear about the target and the border.
Defines the event and risk level.
Validation Key Dependence
Build independent field test set.
Development of assessable outcomes
Understatements are not reported in the calculation of the sub-conditions.
Make sure you decide the next step with the real results.
Test-run returns as a rule.
How do you understand it in the actual business?
The five-second stay of a person in the restricted area is counted as an event, even if successive frames are tested only once; whether a short pass is a warning is determined by the rules of business. The examples do not represent the performance of a particular client, and the actual conclusions need to be verified in conjunction with the enterprise’s own business volume, sample, system and liability boundaries.
The easiest pit to step on.
Report only the overall accuracy rate
I'm gonna use the training video for the check-up.
Frequent alarms without consolidation of incidents
How should we end up receiving and confirming?
The independent test set is misreported by scene, risk and event report and maintains original versions of video, label, model and threshold.
When preparing to communicate with suppliers or internal teams, it is recommended that current processes, representative samples, existing systems, planning time and budget levels be brought. First, the unknown items are clearly marked, and then the decision is made to use diagnostics, PoC, fixed-range projects or ongoing research and development, which is usually more reliable than a direct demand for a price and duration without borders.