Trouble setting up RebuildFullTextIndex

Dec 12, 2013 at 11:45 PM
The company I work at has a site on umbraco and we are trying to set up a job to fully rebuild the index once a week. Almost everything has gone smoothly but when I got to check the webservice it tells me the user does not have access to the server.

Here is what the webservices portion of the config file looks like:

<webservices enabled="True">
    <!-- You must set user-rights for each service. Enter the usernames seperated with comma (,) -->
    <documentServiceUsers>searchindex</documentServiceUsers>
    <fileServiceUsers>searchindex</fileServiceUsers>
    <stylesheetServiceUsers>searchindex</stylesheetServiceUsers>
    <memberServiceUsers>searchindex</memberServiceUsers>
    <templateServiceUsers>searchindex</templateServiceUsers>
    <FullTextServiceUsers>searchindex</FullTextServiceUsers>
    <RebuildFullTextIndexUsers>searchindex</RebuildFullTextIndexUsers>
    <!-- type of files (extensions) that are allowed for the file service -->
    <fileServiceFolders>css,xslt</fileServiceFolders>
</webservices>
Was hoping I could get some direction as to what I needed to do in order to grant the user access to use that service. Thanks in advance!
Dec 13, 2013 at 2:04 PM
Hi D_Creighton

So it seems like with the recent Umbraco web service security updates, the FullTextSearch web service broke. Normally, in UmbracoSettings.config, you needed to enable web services and add the following entry to your web service configuration above:

<maintenanceServiceUsers>searchindex</maintenanceServiceUsers>

However, trying this in Umbraco V6, still returned the error you've mentioned above. I assume you're using one of the V6 versions right?

I tried updating the web service package with the latest version of umbraco.webservices.dll to be compatible with V6, however ran into some issues. I therefore decided to drop the web service and go for the new Umbraco Web API supported in Umbraco 6.1.0+.

Please find the new Web API package here.

In terms of authorisation, it seems that unlike the web service method, this will require a bit more work. I'd like keep it protected to avoid unauthorised usage on your site, but would also like to avoid using basic auth. For now, I've implemented UmbracoAuthorizeAttribute, which means the FullTextSearch Web API calls can only be made within the context of a logged in backend user. More info here. I hope this is suitable for you. Feel free to grab the source code and create your own FullTextSearch Web API controller. This should be fairly easy following these instructions. All FullTextSearch methods for triggering the underlying index functionality is in Governor.Umbraco.FullTextSearch.Admin. The link also describes how you can authorise the Web API calls against a logged in Umbraco Member instead of a User.

The ideal method for authorisation would be to utilise OAuth or something as built-in functionality but I think this can be added as a feature request.

I hope this can point you in the right direction for now.

Best,
Rigardt
Dec 13, 2013 at 3:18 PM
Hi Rigardt.

Sorry, I should have mentioned that we actually use an older version of Umbraco, 4.7.2. I'm guessing that web service won't work either?
Dec 13, 2013 at 3:30 PM
Hi D_Creighton

It might depend on which version of umbraco.webservice.dll you're using and if there's a new version for Umbraco 4.7.2. To test this, can you please add the following line to your umbracoSettings.config for the web services section:

<maintenanceServiceUsers>searchindex</maintenanceServiceUsers>

This is the only web service FullTextSearch is using. You don't need to assign your searchindex user to the others.

The web service definitely works with the original umbraco.webservice.dll (with the security vulnerability). If you're using an updated version, it probably won't work anymore but would be worth testing.

Similar to writing your own web api controller, you can write your own Umbraco web service to call the same FullTextSearch methods. Let me know if the above suggestion solves the issue.

Regards,
Rigardt
Dec 13, 2013 at 3:44 PM
I added that line of code to the umbracoSettings.config and now I don't get any error messages, I just get a blank response. I'm not sure whether that means it's working or not.

I checked the log and it errors out saying object reference not set to an instance of an object.System.NullReferenceException: Object reference not set to an instance of an object so something still isn't working right but it may not be the web service.
Dec 13, 2013 at 3:57 PM
The umbraco.webservices.dll version that I have is 1.04869.18017. Is that the correct version of the dll that I need by chance?

Thanks again for you help with this.

D_Creighton
Dec 13, 2013 at 4:35 PM
Edited Dec 16, 2013 at 9:33 AM
Hi D_Creighton

That is the new umbraco.webservices.dll which is not the one the original FullTextWebservice was written for. A solution would not be to start using the old version again since this has been identified to be vulnerable to security risks. Since the Web API functionality isn't part of Umbraco 4.7.2, we either need to fix the web service to be compatible with the new dll or find another way to call the FullTextSearch methods (e.g. re-indexing etc). After the first look, I had some issues with the new dll, hence rewriting it using Web API. Another option might be to use Umbraco /Base to make external calls available to call the required FullTextSearch methods in Governor.Umbraco.FullTextSearch.Admin (which is all that the original web service did).

For now, I'd suggest having a look at implementing one of these solutions. You basically just need a wrapper for calling the methods in Governor.Umbraco.FullTextSearch.Admin on external HTTP requests. I'll add a request to look at fixing the web service for the new dll but cannot say at this stage exactly how long that will take. You might be able to come up with a solution before I can :)

I hope this helps. Let me know if you have any questions in terms of creating a wrapper for Governor.Umbraco.FullTextSearch.Admin specific to FullTextSearch.

Regards,
Rigardt
Dec 13, 2013 at 6:51 PM
I've never actually made a wrapper at all so anything you could add to help with making one would be greatly appreciated.
Dec 16, 2013 at 9:47 AM
Edited Dec 16, 2013 at 9:53 AM
Hi D_Creighton

You can have a look at the source code (branch 4.7.0 WebService project) for the original FullTextService.asmx.cs. I've included it below. You should be able to reuse the core of this to implement your own Umbraco web service or /Base class. Doing a search online, you should find more info on implementing a web service hosted in Umbraco or a /Base class, either of which can be called externally by your scheduler. The only methods you need from the code below are the AdminActions methods found in Governor.Umbraco.FullTextSearch.Admin to for example re-index the FullTextSearch index etc.

I hope this helps for now.

Regards,
Rigardt

PS: the class below implements umbraco.webservices.BaseWebService which in the current version of the FullTextSearch web service for 4.7.0 still references the old umbraco.webservices.dll. Fixing this issue would involve rebuilding the project against the new version of the dll and dropping Governor.Umbraco.FullTextSearch.WebService.dll in your Umbraco bin. Like I said, feel free to give this a try as you might have it ready before I do :)
using System;
using System.Linq;
using System.Web.Services;
using System.ComponentModel;
using Governor.Umbraco.FullTextSearch.Admin;

namespace Governor.Umbraco.FullTextSearch.WebService
{
    [WebService(Namespace = "http://umbraco.org/webservices/")]
    [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
    [ToolboxItem(false)]
    // Web service that inheits the basic umbraco web service functionality
    // See umbraco docs for how to use this
    public class FullTextService : umbraco.webservices.BaseWebService
    {

        override public Services Service
        {
            get
            {
                return Services.MaintenanceService;
            }
        }
        // Standard umbraco web service method
        [WebMethod]
        public string GetWebservicesVersion(string username, string password)
        {
            // We check if services are enabled and user has access
            Authenticate(username, password);

            var thisVersion = new Version(0, 9);
            return Convert.ToString(thisVersion);
        }
        /// <summary>
        /// Rebuild the entire full text index.
        /// </summary>
        /// <param name="username"></param>
        /// <param name="password"></param>
        [WebMethod]
        public void RebuildFullTextIndex(string username, string password)
        {
            // We check if services are enabled and user has access
            Authenticate(username, password);
            AdminActions.RebuildFullTextIndex();
        }
        /// <summary>
        /// Re-index the supplied list of nodes in the full text index
        /// </summary>
        /// <param name="username"></param>
        /// <param name="password"></param>
        /// <param name="nodes"></param>
        [WebMethod]
        public void ReindexFullTextNodes(string username, string password, int[] nodes)
        {
            // We check if services are enabled and user has access
            Authenticate(username, password);
            if (nodes != null && nodes.Length > 0)
            {
                AdminActions.ReindexFullTextNodes(nodes.ToList());
            }
        }
        /// <summary>
        /// Re-index all nodes in the full text index, but do not delete and rebuld
        /// the entire index as with RebuildFullTextIndex
        /// </summary>
        /// <param name="username"></param>
        /// <param name="password"></param>
        [WebMethod]
        public void ReindexAllFullTextNodes(string username, string password)
        {
            // We check if services are enabled and user has access
            Authenticate(username, password);
            AdminActions.ReindexAllFullTextNodes();
        }
        /// <summary>
        /// /// Re-index the supplied list of nodes and all descendants in the full text index
        /// </summary>
        /// <param name="username"></param>
        /// <param name="password"></param>
        /// <param name="nodes"></param>
        [WebMethod]
        public void ReindexFullTextNodesAndChildren(string username, string password, int[] nodes)
        {
            // We check if services are enabled and user has access
            Authenticate(username, password);
            if (nodes != null && nodes.Length > 0)
            {
                AdminActions.ReindexFullTextNodesAndChildren(nodes);
            }
        }
    }
}