Todd Baginski's Blog - A SharePoint, Mobile, Office, and Azure Blog

HOW TO: Register an External Content Type with the SharePoint Search Service to make the data searchable

Sunset Over Mt. Lincoln

In part 1of this blog post series I showed you how to create a .NET Assembly Connector to read data from a flat file. This blog post shows you how to register an external content type associated with a .NET Assembly Connector with the SharePoint search search service so you can index and search the data coming from the external content type.

The Business Context

As I mentioned in part 1 of this blog post series, it’s commonplace to find critical business data stored inside flat files in large, mature organizations. In addition to being able to browse the data inside web pages, your end users typically want to search for data in the flat file data sources.

The Solution

The SharePoint search service provides the ability to index and search external content types. You may be surprised to see how easy it is to set up the search service to index and search external content sources. This process is very easy to do, in fact it’s really not that much different than it was in the last version of SharePoint. The key methods and properties which allow you to index and search data from an external type are covered in part 1. After you have an external content type registered with the SharePoint 2010 Business Data Connectivity (BDC) service follow these steps to index and search the data.

Create A Content Source

This series of steps builds upon the external content type created in part 1of this blog post series.

  1. Log into your SharePoint Server with credentials that have the permission to administer the SharePoint search service
  2. Click Start, All Programs, Microsoft SharePoint 2010 Products, SharePoint 2010 Central Administration
  3. Click Manage Service Applications
  4. Click the link corresponding to your Search Service Application
  5. In the ribbon, click Manage (see screenshot below)
    Search Service Application
  6. When the page refreshes, click Content Sources
  7. When the page refreshes, click New Content Source
  8. In the name textbox enter Flat File Data
  9. Select the Line of Business Data Content Source Type radio button
  10. When the page refreshes, select the Business Data Connectivity service running on your server in the Business Data Catalog Service Application dropdown list
  11. Select the Crawl selected external data source radio button
  12. Check the checkbox next to the FlatFileLOBSystemInstance external data source
  13. Check the Start full crawl of this content source checkbox
  14. Click OK
    image
  15. When the page refreshes, click the Refresh link until the Status for the Flat File Data Content Source says Idle
    Flat File Data Content Source Status

Create A Search Scope

After creating the content source, create a new search scope to make the content source searchable.

  1. Log into your SharePoint Server with credentials that have the permission to administer the SharePoint search service
  2. Click Start, All Programs, Microsoft SharePoint 2010 Products, SharePoint 2010 Central Administration
  3. Click Manage Service Applications
  4. Click the link corresponding to your Search Service Application
  5. In the ribbon, click Manage
    Search Service Application
  6. When the page refreshes, click Scopes
  7. When the page refreshes, click New Scope
  8. In the Title textbox enter Flat File Data Source
  9. Click OK
  10. When the page refreshes, click the Add rules link next to the Flat File Data Source search scope
    Add Rules Link
  11. In the Scope Rule Type section, select the Content Source radio button
  12. In the Content Source dropdown select Flat File Data
  13. In the Behavior section select the Include – Any item that matches this rule will be included, unless the item is excluded by another rule radio button
  14. Click OK
    Create Scope
  15. When the page refreshes, click Search Administration
  16. When the page refreshes, click Start update now
    Update Scope
  17. Refresh the page until the scope update process completes
    Scope Update Process Complete

Create A Search Center Site

This example assumes a search center site must be created on your SharePoint server. If you already have one created you can skip to the next series of steps to configure the search scopes dropdown list

  1. Open your SharePoint web site in a web browser
  2. Click Site Actions
  3. Select New Site
  4. Click the Search tab

    Note: The Search tab may or may not be available, depending on the template your SharePoint site is based upon. In this example my SharePoint site is a team site, which supports the creation of Search Center sub sites.

  5. Select the Basic Search Center template
  6. In the Name textbox enter Search
  7. In the URL textbox enter search

    Create Basic Search Center

  8. Click Create

Configure the search scopes dropdown list

  1. After the Basic Search Center site is created click Site Actions
  2. Select Site Settings
  3. Click Go to top level site settings
  4. Click Search scopes
  5. Select the Use custom scope radio button
  6. Enter the path to your Basic Search Center sub site

    Note: In my example the search center site is a sub site two levels deep, under the sites URL. That’s why you see /sites/team/search in the screenshot below.

    Search Scope Settings

  7. Select the Show scopes dropdown option in the Site Search Dropdown Mode dropdown list
  8. Click OK
  9. Click Search scopes
  10. Click Display Groups
  11. Click Search Dropdown
  12. Check the Flat File Data Source Scope Name
  13. Click OK

    Configure Scopes Dropdown List

Search For Data In The External Content Type

  1. Open your SharePoint site in a web browser
  2. Select the Flat File Data Source search scope in the search scope dropdown list
  3. Enter a search term that will return a result from the external content type
  4. Click the hourglass icon

    Search For A Term

  5. View the search results

    Search Results

15 responses on “HOW TO: Register an External Content Type with the SharePoint Search Service to make the data searchable

  1. decatec says:

    thanks again for your excellent review

  2. Todd says:

    Hi Sanky,

    Yes, there is. You can stream any blob as long as the back end you are accessing supports a streaming interface. Then, based on the MIME type you can reconstruct the blob.

    See this article for more details:

    http://msdn.microsoft.com/en-us/library/ee556429(office.14).aspx

  3. Tommy Kristoffersen says:

    Hi Todd!

    Great article!

    Instead of search url "bdc://flatfile….", do you have any idea how you could use "DisplayUriField" to point to your own custom search url?

    I found this property in the SDK, but can’t get it to work. The search url of "bdc://flatfile…." gives no meaning.

  4. bmm6o says:

    Great series of articles. I think I followed your guide to the letter, but my crawl is failing. I get the error "Error while crawling LOB contents. ( Access denied by Business Data Connectivity. )" for the url "bdc3://flatfilelobsystem_flatfilelobsysteminstance/…". I gave read rights to everyone, but still get the error. Any ideas?

  5. bmm6o says:

    Great articles.

    I can create the content source, but I get this error in the crawl log:

    "bdc3://flatfilelobsystem_flatfilelobsysteminstance/default/00000000%252d0000%252d0000%252d0000%252d000000000000/824/flatfilelobsysteminstance/826&s_ce=048u0h3o40800080g1020008hs200

    Error while crawling LOB contents. ( Access denied by Business Data Connectivity. ) "

    I went to the permissions page for the BDC external content type, and gave all permissions to "everyone" and "All Users", but still the same error. Is there a specific account I need to give permissions to? What is it?

    Thanks.

  6. Rahul Khedekar. says:

    Thanks for great article.

    I have an external system which can expose contents through web service. When user searches something in SharePoint, I want to search this external system as well and want to combine search results with SharePoint Search result. My external system can expose contents in it using web service and using BCS I can index it as you discussed.

    But problem is my external system has it’s own permission check model. Is there way I can apply this permission checks to search results? I mean if logged in user doesn’t have permssion to view some data then that won’t be includede in search results.

    Furthermore, My external system also has very good search services which does permission check before showing data in search result. If I use this service to search my external content then permission thigs are already handleded in the search service itself.

    So considering this scenario, Is there any way I can configure SharePoint Search to utilize my search service? Or if only BCS is answer to search external data, is there any way I can filter results based on permission logic in external system?

    Thanks,
    Rahul

  7. Greg Robinson says:

    I have created a .NET connector assembly that pulls data from an in-house .asmx web service. I have successfully created an external list in a search center site that displays this data without problem.

    However, I’ve run into an error and a warning when indexing this list in search server 2010 for which I cannot find much information. I’ve configure a content source with LOB data as the content source type and select my external data source from the list.

    After running a full crawl, I receive this error:

    Error while crawling LOB contents. ( Not a valid Win32 FileTime. )

    And this warning:

    The content for this address was excluded by the crawler because this item was marked with a no-index meta-tag. To index this item, remove the meta-tag and recrawl.

    Any ideas on how to resolve this?

  8. GravatarI know, that on youtube there is a video, where the creator explains all the details of the preferences.

  9. Susan says:

    Thanks for the tutorial! Now we can manage it easily. adjustable beds

  10. This post is exactly what I am interested. keep up the good work. we need more good statements.

  11. Nice says:

    I have seen your older website but i like it because you can say it is enhancement of older. Here i have read your article "Percentage of Deceased Persons who are Cremated by State". This is highly informatics, crisp and clear. Thanks!!!Nanny Jobs London

  12. K. Meena says:

    Hi:

    The post is an extremely useful one. I could follow all the steps clearly.

    I am facing a problem when Full crawl is performed. The error in the crawl log reads as:
    Error while crawling LOB contents. ( Value cannot be null. Parameter name: fieldDotNotation ).

    I am clueless about how to solve this problem. Any help here would be highly appreciated.

    Regards
    Meena K

  13. Andy says:

    Todd,

    Thanks for the great tutorial.
    Anyway, I would like to ask you something.

    According to the search result page, the search result icon is all "Folder" icon. But, I want the icon to be specific according to the file such as word, pdf, excel document.

    Is it possible to do that?
    Thanks

  14. Ashwani says:

    Hi There,

    we were able to create an external content type & were also able to crawl the content.
    the issue we have it that all the records of the external list are returned as 1 single search result.
    we want a functionality where we would like to have the recors in the result set a individual records as they are in the external list.
    So if the list has 10 records & my search term matched 5 records, i should get 5 documents in the result set.
    we have done the mapping of the external list columns with the required managed properties.

    Any Help on this would be greatly appreciated.

    Thanks,
    Ashwani

  15. EDDY says:

    how can i perform the same process using custom indexing connector??