In part 1of this blog post series I showed you how to create a .NET Assembly Connector to read data from a flat file. This blog post shows you how to register an external content type associated with a .NET Assembly Connector with the SharePoint search search service so you can index and search the data coming from the external content type.
The Business Context
As I mentioned in part 1 of this blog post series, it’s commonplace to find critical business data stored inside flat files in large, mature organizations. In addition to being able to browse the data inside web pages, your end users typically want to search for data in the flat file data sources.
The Solution
The SharePoint search service provides the ability to index and search external content types. You may be surprised to see how easy it is to set up the search service to index and search external content sources. This process is very easy to do, in fact it’s really not that much different than it was in the last version of SharePoint. The key methods and properties which allow you to index and search data from an external type are covered in part 1. After you have an external content type registered with the SharePoint 2010 Business Data Connectivity (BDC) service follow these steps to index and search the data.
Create A Content Source
This series of steps builds upon the external content type created in part 1of this blog post series.
- Log into your SharePoint Server with credentials that have the permission to administer the SharePoint search service
- Click Start, All Programs, Microsoft SharePoint 2010 Products, SharePoint 2010 Central Administration
- Click Manage Service Applications
- Click the link corresponding to your Search Service Application
- In the ribbon, click Manage (see screenshot below)
- When the page refreshes, click Content Sources
- When the page refreshes, click New Content Source
- In the name textbox enter Flat File Data
- Select the Line of Business Data Content Source Type radio button
- When the page refreshes, select the Business Data Connectivity service running on your server in the Business Data Catalog Service Application dropdown list
- Select the Crawl selected external data source radio button
- Check the checkbox next to the FlatFileLOBSystemInstance external data source
- Check the Start full crawl of this content source checkbox
- Click OK
- When the page refreshes, click the Refresh link until the Status for the Flat File Data Content Source says Idle
Create A Search Scope
After creating the content source, create a new search scope to make the content source searchable.
- Log into your SharePoint Server with credentials that have the permission to administer the SharePoint search service
- Click Start, All Programs, Microsoft SharePoint 2010 Products, SharePoint 2010 Central Administration
- Click Manage Service Applications
- Click the link corresponding to your Search Service Application
- In the ribbon, click Manage
- When the page refreshes, click Scopes
- When the page refreshes, click New Scope
- In the Title textbox enter Flat File Data Source
- Click OK
- When the page refreshes, click the Add rules link next to the Flat File Data Source search scope
- In the Scope Rule Type section, select the Content Source radio button
- In the Content Source dropdown select Flat File Data
- In the Behavior section select the Include – Any item that matches this rule will be included, unless the item is excluded by another rule radio button
- Click OK
- When the page refreshes, click Search Administration
- When the page refreshes, click Start update now
- Refresh the page until the scope update process completes

Create A Search Center Site
This example assumes a search center site must be created on your SharePoint server. If you already have one created you can skip to the next series of steps to configure the search scopes dropdown list
- Open your SharePoint web site in a web browser
- Click Site Actions
- Select New Site
- Click the Search tab
Note: The Search tab may or may not be available, depending on the template your SharePoint site is based upon. In this example my SharePoint site is a team site, which supports the creation of Search Center sub sites.
- Select the Basic Search Center template
- In the Name textbox enter Search
- In the URL textbox enter search
- Click Create
Configure the search scopes dropdown list
- After the Basic Search Center site is created click Site Actions
- Select Site Settings
- Click Go to top level site settings
- Click Search scopes
- Select the Use custom scope radio button
- Enter the path to your Basic Search Center sub site
Note: In my example the search center site is a sub site two levels deep, under the sites URL. That’s why you see /sites/team/search in the screenshot below.
- Select the Show scopes dropdown option in the Site Search Dropdown Mode dropdown list
- Click OK
- Click Search scopes
- Click Display Groups
- Click Search Dropdown
- Check the Flat File Data Source Scope Name
- Click OK


thanks again for your excellent review
Hi Sanky,
Yes, there is. You can stream any blob as long as the back end you are accessing supports a streaming interface. Then, based on the MIME type you can reconstruct the blob.
See this article for more details:
http://msdn.microsoft.com/en-us/library/ee556429(office.14).aspx
Hi Todd!
Great article!
Instead of search url "bdc://flatfile….", do you have any idea how you could use "DisplayUriField" to point to your own custom search url?
I found this property in the SDK, but can’t get it to work. The search url of "bdc://flatfile…." gives no meaning.
Great series of articles. I think I followed your guide to the letter, but my crawl is failing. I get the error "Error while crawling LOB contents. ( Access denied by Business Data Connectivity. )" for the url "bdc3://flatfilelobsystem_flatfilelobsysteminstance/…". I gave read rights to everyone, but still get the error. Any ideas?
Great articles.
I can create the content source, but I get this error in the crawl log:
"bdc3://flatfilelobsystem_flatfilelobsysteminstance/default/00000000%252d0000%252d0000%252d0000%252d000000000000/824/flatfilelobsysteminstance/826&s_ce=048u0h3o40800080g1020008hs200
Error while crawling LOB contents. ( Access denied by Business Data Connectivity. ) "
I went to the permissions page for the BDC external content type, and gave all permissions to "everyone" and "All Users", but still the same error. Is there a specific account I need to give permissions to? What is it?
Thanks.
Thanks for great article.
I have an external system which can expose contents through web service. When user searches something in SharePoint, I want to search this external system as well and want to combine search results with SharePoint Search result. My external system can expose contents in it using web service and using BCS I can index it as you discussed.
But problem is my external system has it’s own permission check model. Is there way I can apply this permission checks to search results? I mean if logged in user doesn’t have permssion to view some data then that won’t be includede in search results.
Furthermore, My external system also has very good search services which does permission check before showing data in search result. If I use this service to search my external content then permission thigs are already handleded in the search service itself.
So considering this scenario, Is there any way I can configure SharePoint Search to utilize my search service? Or if only BCS is answer to search external data, is there any way I can filter results based on permission logic in external system?
Thanks,
Rahul
I have created a .NET connector assembly that pulls data from an in-house .asmx web service. I have successfully created an external list in a search center site that displays this data without problem.
However, I’ve run into an error and a warning when indexing this list in search server 2010 for which I cannot find much information. I’ve configure a content source with LOB data as the content source type and select my external data source from the list.
After running a full crawl, I receive this error:
Error while crawling LOB contents. ( Not a valid Win32 FileTime. )
And this warning:
The content for this address was excluded by the crawler because this item was marked with a no-index meta-tag. To index this item, remove the meta-tag and recrawl.
Any ideas on how to resolve this?
GravatarI know, that on youtube there is a video, where the creator explains all the details of the preferences.
Thanks for the tutorial! Now we can manage it easily. adjustable beds
This post is exactly what I am interested. keep up the good work. we need more good statements.
I have seen your older website but i like it because you can say it is enhancement of older. Here i have read your article "Percentage of Deceased Persons who are Cremated by State". This is highly informatics, crisp and clear. Thanks!!!Nanny Jobs London
Hi:
The post is an extremely useful one. I could follow all the steps clearly.
I am facing a problem when Full crawl is performed. The error in the crawl log reads as:
Error while crawling LOB contents. ( Value cannot be null. Parameter name: fieldDotNotation ).
I am clueless about how to solve this problem. Any help here would be highly appreciated.
Regards
Meena K
Todd,
Thanks for the great tutorial.
Anyway, I would like to ask you something.
According to the search result page, the search result icon is all "Folder" icon. But, I want the icon to be specific according to the file such as word, pdf, excel document.
Is it possible to do that?
Thanks
Hi There,
we were able to create an external content type & were also able to crawl the content.
the issue we have it that all the records of the external list are returned as 1 single search result.
we want a functionality where we would like to have the recors in the result set a individual records as they are in the external list.
So if the list has 10 records & my search term matched 5 records, i should get 5 documents in the result set.
we have done the mapping of the external list columns with the required managed properties.
Any Help on this would be greatly appreciated.
Thanks,
Ashwani
how can i perform the same process using custom indexing connector??