A opportunity for patent informatics today is to benefit from the research of the Information Retrieval (I/R) community in order to provide more substance to the claims in new technical advances.
Information Retrieval uses Precision and Recall in order to asses different approaches; these have to be measured against specifically agreed benchmarks, which is a typical requirement for every serious activity.
Precision and Recall exhibit opposite trends, since by augmenting the number of retrieved documents, the number of search related documents increases, but the percentage of relevant documents decreases
The objective of new algorithms it to provide a better performance both in Precision as well in Recall ; this can be accomplished for example by using sequential methods.
In summary, patent information retrieval is a specific and well characterized area of document information retrieval, due to predefined structure of patent documents, to the specific language used and to the different subcases of patent searching, which are characterized by different costs of missing important information, hence by a different balance between Precision and Recall.
The most significant and specific activities today known for identifying patent I/R benchmarks are carried out by NTCIR in Japan, which regularly collects, updates and experiments Information Retrieval test cases for different tasks, some of which are patent specific, though focused to far eastern languages (Japan, China, South Korea) besides the English one.
Other than this, NTICR collects benchmarks in patent classification (or mapping), i.e. in the refinement of a list of retrieved patents, classified by problem solved and approach used, which can be also considered a very related topics.
Other kind of activities or at least plans can also be found in the web, and must be of course encouraged.
Some more information can be found in the full presentation, which can be accessed from http://www.intellipatent.eu/Documents/PatentInformatics2.pdf
