We have over 1,000 pdf documents on our website, and the most important is our Journal, with 238 editions at present. As an example, a couple of issues of our Journal are here, but they are all similar in structure:
You will see that at the bottom of the second page of Journal 100 is a table of contents for the issue and, if the pdf is saved and then opened in Acrobat Reader, there is an equivalent set of bookmarks. It would help if a search term appearing in the article title (as listed in the table of contents) was given a higher weighting than one in the text of the document, but quite often the term will only occur in the text, and not in the title at all.
Ideally, we would like the weighting to be based upon articles, but weightings based upon the the whole Journal is acceptable. This is because the articles tend to be on unrelated topics within a Journal, so there is probably little difference between the number of instances of a specific search term within an article, and within its parent document.
I think it would help me if you could explain, in non-technical terms, how the four weightings operate with WPFTS. I've looked at the TFIDF article on Wikipedia, and understand the basics, but the majority of the article is too technical for me. Perhaps this information could also be added to the WPFTS documentation?
As a related issue, could WPFTS open the document listed in the search results at the first article page where the search term is found? I realise it might go to the article title instead, and if so that wouldn't really help much.
Search Results - BOOLEAN Operators and Relevance
Our implementation of WPFTS is configured with the Default Search Logic set to "AND".
I did a search of our website for "2015 Committee", both with and without the quotation marks (the results were the same), and the second and third most relevant results did indeed contain the phrase "2015 Committee"; the second twice and the third once
However, the top result did not. It was a Bibliography for the year 2015, and contained "Committee" nine times, and "2015" 962 times, but in completely different parts of the document.
This feels more like an "OR" result than an "AND" one, but maybe I misunderstand how the "AND" and "OR" operators work?
EpsilonAdmin last edited by
Yes, the algorithm thinks the number of words here is more important than the exact phrase... Ideally, if we could justify the relative weight for the phrase matches. Currently, it's too low for your case, but I think if we can double it, this could solve the issue.
I think to put this parameter to settings in the near future. Thanks!