Categories

• Announcements

  Announcements regarding our community
  1 Topics

  1 Posts

  E

  A new version of WPFTS with a fundamentally new search algorithm is out now!

  Some time ago, we have accumulated quite a lot of feedback from users who very much asked to add an improvement in search - to significantly increase the search speed on a large number of documents and add the ability to search for a phrase. This became somewhat of a challenge for us since it seemed almost impossible to implement a full-fledged indexed search using the small power of PHP and MySQL.

  New Search Algorithm

  But, as a result, we managed to get closer to the goal. To achieve this, the search algorithm had to be completely redone. Yes, it's still TF-IDF, but with significant additions to the logic and relevance calculation. In particular, we have added a "phrase bonus", which allows those documents in which similar phrases were encountered to be displayed in higher places. Note that phrases, in this case, can be not only EXACT matches but also partial matches of words, as well as nearby words (not only close ones).

  For example, for the search phrase "infinite space", the relevance will be increased in those documents where the phrases "infinite distant space" and even "space seems to be infinite" are found. The word order in the phrase is currently ignored. An option that includes strict word order in the phrase will appear in the next updates.

  Deep searching is faster

  Also, the speed for "deep search" has been significantly increased. So, for example, by default the query "cat" will only find words like "catty", "catalog", that is, those whose beginning exactly coincides with the search word. When you enable "deep search", the words "subcategory", "procatering", "mystification" and the like will also be found, that is, words in which the "cat" occurs in the middle or at the end.

  Previously, a "deep" search literally put the algorithm into a stupor and the search could take 10, 20, or even more seconds. Now, with our new algorithm, deep search has become much faster, and even on a large number of documents can take only 2-3 times longer than a regular "non-deep" search.

  No character limit anymore

  Thanks to the new algorithm, it becomes possible to get rid of the 3 character limit on the length of the request. Previously, this limitation was necessary because searching for one or two characters set in motion almost the entire search index, as a result of which the query could run for a very long time and even lead to a script crash. Now it has become possible to remove the limitation on the number of characters since requests of one or two characters only receive a limited number of matches and do not significantly slow down or crash the script.

  Faster with the Word Index

  We have introduced one more small index - word index. It allows you to increase the search speed significantly for a large number of documents, since it partially relieves the load on MySQL, eliminating the need to perform concatenation operations on huge tables.

  MyISAM support was dropped

  We have dropped support for the MyISAM table type and removed the corresponding option and all associated algorithms from the code. Tables of this type all too often lead to "locking" and despite the fact that they sometimes give a 10-20% bonus to search speed, people decided to abandon MyISAM in favor of the more modern type as InnoDB.

  Faster re-indexing

  Indexing and re-indexing of documents have also been significantly accelerated, although the new "word index" slightly increases the re-indexing time. We are still working on improving this part of the plugin.

  Work on improving the plugin is very active, only about 40% of all planned improvements for this year have been implemented. Therefore, your feedback and understanding that bugs may occur are very important to us. If you notice an error or inaccuracy, please report it immediately on the plugin support forum.

  Let me remind you that I have a Patreon page. If you want to support the development of this awesome plugin, I would be very grateful.

  Thank you for using WPFTS!

• Frequently Asked Questions

  Got a question? Ask away!

  Troubleshooting Guide
  22 Topics

  48 Posts

  E

  Hi @Simply_Friend

  Thank you for the question. Actually it's not planned to be added to the plugin core, because this feature (search by date FROM - TO) is already implemented in WP Native search (in WP_Query()).

  So you can search with date using WP_Query parameter date_query.

  Here is the topic: https://developer.wordpress.org/reference/classes/wp_query/#date-parameters

  Another question is if you want date picker fields to be added next to the text query field in the search widget.

  This is fairly easy to do for the average WP developer. There are probably some visual constructors to do this, but I can’t recommend anything here - if you find a solution, please write. It will be useful for other users.

  We don't plan to add a search form builder to the core functionality of WPFTS because we don't want to compete with Elementor / Beaver Builder / Visual Builder etc etc, although we will probably do a detailed tutorial on how to use the builders to make a search widget with additional fields.

  Hope this helped you.

  Thanks!

• Bugs and Fixes

  Noticed a bug or an issue? The solution is (probably) here!
  47 Topics

  237 Posts

  E

  Hi @stark

  Unfortunately, WPFTS does not work with PODS custom fields "from the box", so custom code exists somewhere. I think your developer used another way to add data to the search index.

  We need to ask him or alternatively, I could check your code myself, it should not be too long. In this case please send me admin access to your resource using a private message.

  Thanks.

• Recipes and Known Solutions
  36 Topics

  91 Posts

  E

  In fact, the problem can sometimes be caused not by a slow search by WPFTS plugin or even by a slow generation of Smart Excerpts. The search slowdown was caused by a bug (or flaw) in the code of the Divi theme itself.

  Let's get closer to the code. So, one of the clients contacted us with a complaint that some search requests took too long to process (about 10-12 seconds), which is extremely non-standard for WPFTS even with 500k records, and the client at that time had less than 1000 records. The problem was detected on WPFTS Pro v2.51.212.

  After spending several hours deeply studying the code, we found out that due to a flaw in the code of the Divi Theme, post previews are generated anew each time when displaying search results, while logically they should be generated only once and then used.

  There is indeed such a check in the code, but it only works for those images that are smaller in size along one of the axes (height or width). If the image is larger than the target size of the preview, then the code for checking the presence of an already generated preview is simply not called, and the preview is generated again.

  This spends 1-2 seconds for each post in the output results, which, with a number of 10 posts, gives a noticeable slowdown of 10-15 seconds.

  Below we have attached a new version of the Divi Theme file which is suitable for version 4.25.2. We have already reported the bug to the authors of this topic, so we hope that it will be fixed in future versions.

  We hope this article helped you.
  DIVI_THEME_FIX.ZIP

• Blogs

  Blog posts from individual members

  0 Topics

  0 Posts

  No new posts.
• General Discussion

  A place to talk about whatever you want
  14 Topics

  42 Posts

  E

  Hi @marcellosalass

  Actually WPFTS could be not that great for you in case you have millions of records (e.g. posts).
  The whole Wordpress could be not good for you either, because to handle such amount of data you will need to use specific optimization and caching algorithms and even clustering.

  I would like to propose you to use full-text solutions based on Apache Solr or Elastic Search - those tools are implemented in Java and work way faster than PHP+MySQL solution provided by WPFTS and they are especially good for big data.

  Alternatively you can wait until we release a new WPFTS version (in 1-2 months) that can use self-hosted Elastic Search engine to keep full-text index and uses WPFTS data collecting algorithms.

  Thank you for the question.