30 December 2010

SharePoint Check In, Check Out, Content Approval, Draft Item Security and Version History Explained



There seems to be much confusion out in the SharePoint world around the topics of Check In/Out, Content Approval, Version History and Draft Item Security. My goal is to explain and clarify how all these function together.

REQUIRE CHECK OUT, CHECK OUT, CHECK IN

It all starts with the creation of a new document be it created via the New menu or uploaded. When a user creates or uploads the new document, if “Require Check Out” is enabled, SharePoint will check out the document for the user automatically. Nobody, not even Approvers, can see the document until it is checked in.

Approvers can NEVER see content that is NOT checked in!

This is the function of check out and check in. It is a way for a user to work on a document in private until such time they are ready to share the document with others. When they then check it in, it moves to the next level of approval. If a situation unfolds where the user leaves the company and has a document still checked out, an Administrator can either do a “Discard Check Out” or “Take Ownership” can be done followed by a Check In.

This gets tricky in the case of a newly uploaded document. Because Approvers cannot see the document, the user doing the upload needs to be aware of this because they have to do the Check In to allow Approvers to see the document. It’s the same as the first iteration of a New document. Nobody except the author knows that the document actually exists… at least not until Check In.

In cases where users do mass uploads, having the “Force Check Out” option selected on the document library, can cause major problems as the uploading user would have to approve each document individually. That can be a very time consuming process. I always recommend that in such cases, the Administrator temporarily disable the “Force Check Out” option to allow the library content to be populated. With Content Approval turned on, this temporary change, would not cause major issues.

CONTENT APPROVAL

Upon check in, if “Content Approval” is turned on, NOW Approvers as well as the Author can see the document. Normal users cannot see the document until it is approved.

If Content Approval is not turned on, checking in the document, would make it visible to everyone. With Content Approval turned on, an approver would need to review the document and then approve it in order to make it visible to everyone.

DRAFT ITEM SECURITY

When the Draft Item Security option is enabled, it usually deals with a scenario where you may have a collaborative group working on a document. This usually ties in with a Version History setting that has both Major (1.0, 2.0, 3.0 etc.) and Minor (1.1, 1.2, 1.3 etc.) versions. In such a case, there may be a group of people that have Member/Contributor rights that allows them to change document. There are some users who are Approvers and setting this setting determines who of these users can see items AFTER they are checked in. Remember that only the active author can see items that are checked out.

This option is only available when the Version History setting is set to use Minor versions.

VERSION HISTORY

Lastly there is Version History. Version history works independently from the above items with the exception of Draft Item Security which is only available as an option when Version History is set to Minor versions.

When set to Major versions, each update to the document results in a new version e.g. 1.0, 2.0, 3.0 etc.

When set to Minor versions, each update results in a new minor version e.g. 1.1, 1.2, 1.3 etc. When the document is then Approved, a new major version is created. As an example, the document is worked on and checked in by Bob, resulting in version 1.1. Now Sue edits the document and upon check in, it results in version 1.2. This continues until they are ready for the document to be approved. At this point, they choose to Publish a Major version. The document approvers are notified of the submission and they review it. Once they’re happy with it and approve the document, SharePoint publishes the document as version 2.0.

And that is how these items work together in the SharePoint environment. I hope this helps to shed some light on this subject.



Cheers
C

29 December 2010

SharePoint 2010 People Picker de-mystified!

The problem with most demos and classes is that the presenters use straight Active Directory for the configurations and though this most certainly has the desired effect of making the demo run smoothly because everything “just works”, it doesn’t help you and me in the REAL WORLD where more often that not, we encounter a blended environment.  If you are fortunate enough to work in a straight AD shop, count your blessings every night!
For the rest of us, we have to make that “blend” work for the customer because at the end of the day, they don’t care, nor should they, about the technical difficulty behind the scenes to make it work… they just want it to work… first time… every time. 🙂
Such is the case when you’re dealing with a Forms Auth/LDAP environment.  My client has a huge deployment in SP2007 under LDAP.  We deployed SP2010 under Claims Auth which meant a shift away from our old SiteMinder LDAP authentication scheme.  Of course, under Forms Auth, we controlled the database against which the authentication and searching was done ala the source against which the PeoplePicker control was searching.  Under claims, AD is the source and that adds new technical challenges to the mix in order to ensure that your customer search experience remains the same.  Thus my search for information about the PeoplePicker in 2010 began…
There are lots of links out there related to PeoplePicker and custom AD searches.  The first and most obvious is this MSDN link that gives us the basic overview.  It even provides some basic examples, but it doesn’t actually EXPLAIN how to use it.  Joel posted on his old blog on the topic, but it was basically a collection of what’s on MSDN so I was still looking for more detail.  There were lots of other posts on the “peoplepicker-searchadcustomquery” switch in STSADM, but they were all either references to the MSDN article or copy and paste jobs. 🙁
In the end, I was left to decipher the problem the old fashion way… through trial and error.  After many attempts, I finally managed to get a good read on how it actually works and was able to explain it to my colleagues as well.  Now I’m hoping this blog post saves someone some time in the future when they have to work through this.  I’ll begin with my Powershell script:
 1:  clear-host
   2:  write-host -f green "========================="
   3:  write-host -f green "========= BEGIN ========="
   4:  write-host -f green "========================="
   5:  write-host ""
   6:  $snapin = "Microsoft.SharePoint.PowerShell"
   7:  if (get-pssnapin $snapin -ea "silentlycontinue")
   8:  {
   9:    write-host -f green "PSsnapin $snapin is loaded"
  10:    write-host ""
  11:  }
  12:  else
  13:  {
  14:    if (get-pssnapin $snapin -registered -ea "silentlycontinue")
  15:    {
  16:      write-host -f green "PSsnapin $snapin is registered"
  17:      Add-PSSnapin $snapin
  18:      write-host -f green "PSsnapin $snapin is loaded"
  19:      write-host ""
  20:    }
  21:    else
  22:    {
  23:      write-host -f Red "PSSnapin $snapin not found"
  24:      write-host ""
  25:    }
  26:  }
  27:  [void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint")
  28:  $webapps = get-spwebapplication
  29:  foreach ($webapp in $webapps)
  30:  {
  31:    write-host -f yellow $webapp.displayname;
  32:    write-host -f yellow $webapp.url;
  33:    stsadm -o setproperty -url $webapp.url -pn peoplepicker-searchadcustomquery -pv "(userPrincipalName={0}*)(givenName=*{0}*)(sn=*{0}*)(displayName=*{0}*))";
  34:    stsadm -o setproperty -url $webapp.url -pn peoplepicker-searchadforests -pv "domain:<<>>.com";
  35:  }
  36:  write-host ""
  37:  write-host -f green "========================="
  38:  write-host -f green "========== END =========="
  39:  write-host -f green "========================="
  Lines 1-5 simply clears the screen and writes a starting header. Line 6 sets a local variable ($snapin) to “Microsoft.SharePoint.PowerShell”. Lines 7-11 checks if the snapin is loaded already.  If it is, it writes a message on screen noting that.
Lines 12-26 kicks in if the snapin is not already loaded.  In this case, it gets the registration info for the snapin and then tries to load the snapin.
Line 27 loads the Microsoft.SharePoint.dll in order to access the SharePoint object model. Line 28 sets a local variable ($webapps) to the return value of the “get-spwebapplication” PowerShell cmdlet.
Lines 29-30 sets up an iteration loop for each object in the $webapps variable.  It temporarily sets the current object to another local variable ($webapp) for processing in subsequent lines.
Lines 31-32 writes the Display Name and the URL values of the web application being processed.
Lines 33-34 is where the magic happens.  These lines call STSADM and passes the needed values to it in order to set SharePoint’s metadata properties for the people picker.
Line 35 closes the loop.
Lines 36-39 simply writes to closing a closing footer.
Now let’s look at the two STSADM commands in more detail.
stsadm -o setproperty -url $webapp.url -pn peoplepicker-searchadcustomquery -pv "(userPrincipalName={0}*)(givenName=*{0}*)(sn=*{0}*)(displayName=*{0}*))";
The key is the “setproperty” operation in STSADM.  Since these properties are stored by SharePoint on a web application basis which is why this command is issued for each web application in SharePoint.
The first switch we provide to the operation is the URL of the web application.
The second switch is the property name and in this case, we’re targeting the “peoplepicker-searchadcustomquery” property.
The third switch is the property value.  The value we pass here is the search values that is appended and passed into Active Directory for the search query.  This is where the available documentation is lacking.  The format to use is as follows:
(<AD Name>=<wildcard>{0}<wildcard>)
where <AD Name> is replace by the Active Directory property name you’re targeting and <wildcard> is replaced by either a star (*) if you want to wildcard on that side of the query or nothing if you don’t want the wildcard on that side.
In our example, we are targeting 4 Active Directory properties.  The first is “userPrincipalName” which is the user ID e.g. “cjvandyk@crayveon.com”.  In this example, we are using a wildcard at the back of the query so a search such as just using “cjvandyk” or “cjvan” should yield a match.
The second property is “givenName” which is a user’s fist name.  We are using front and back wildcards here so any part of the user’s name would yield a match.
The third property is “sn” which is the user’s last name.  Again, we’re using front and back wildcards in order to yield a match on any part of the user’s last name.
The last property is “displayName”.  This field is populated with the commonly used name of the user.  In most cases, a user’s name is registered for a match on the default query passed to Active Directory as “First MI Last” for example, a user would be matched as “Robert E. Doe” within the system.  For a user to locate him in a people picker they’d have to use “Robert E. Doe”.  In person he may be known as “Bob Doe” instead.  A user trying to find Bob in the people picker with “Bob Doe” won’t get a match, but if the Active Directory property for “displayName” is populated with this value i.e. “Bob Doe”, our fourth property would yield a match for the user.
The second STSADM property we are setting for the web application is that of the “peoplepicker-searchadforests”.  This is used when you have a very large Active Directory and wish to limit the AD query down to just a specific sub set instead of searching the entire directory.  Doing this, can result in some major performance gains in your people picker.  The syntax is:
stsadm -o setproperty -url $webapp.url -pn peoplepicker-searchadforests -pv "domain:<<<CHANGEME>>>.com";
As before, our first switch passed is the URL of the target web application.
The second switch is again the property name which as we’ve already mentioned is “peoplepicker-searchadforests”.
The third switch is the property value.  In this switch, simply replace the “<<<CHANGEME>>>” with your target domain you wish to limit the search to.
The last thing you need to do, is ensure that your Active Directory team configure indexes on all the properties that you have specified in your search query.  Having indexes on these properties within AD will also give you a good boost in the performance of the people picker control.
Provided that your AD is populated correctly, using this script, you can configure the people picker for optimal results for the end user.  For your convenience, you can download the PowerShell script from my downloads library located here:


Cheers
C

14 December 2010

BUG – Security concern – Overriding library permissions breaks previously configured item level permissions in SharePoint 2007

When you are having to override library permissions where you have previously overridden folder permissions and/or item permissions to break inheritance, BE WARNED that permissions behavior in SharePoint may not be what you’d expect. I’ll explain by demo… Start by navigating to the target library.image_50_7E4473DANow go ahead and override item permissions. Upon completion, you should have new permissions for the item.image_80_4DD09FAC Now navigate back to the top site. Go ahead and override library permissions. In my case, I removed Members and Visitors from the library level permissions.image_83_427A00BANow navigate back to the item again. You’ll notice some permissions missing.image_86_427A00BAThe conclusion is that breaking inheritance at the library level will override the item level permissions and cause unexpected results. This can be particularly bad if you’re unaware of this behavior and you have an external facing site that has had item level permissions configured for certain external partners and someone breaks the inheritance on the library level as it could grant access to content to unintended users. BE WARNED!!!

Cheers
C

03 December 2010

Latest interesting SharePoint 2010 downloads

As expected, the avalanche of documentation, management packs, SDKs etc. for SharePoint 2010 is now starting to overrun the landscape.  Here are the latest and most interesting ones:
                                   

Cheers
C

02 December 2010

Install the KB982307 hotfix as a prerequisite for PowerPivot

NOTE:  This KB hotfix has to be installed on all web front end as well as application servers in your SharePoint farm.
  1. Begin by downloading the applicable architecture (x86/x64/i64) version for your environment from here: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=3e102d74-37bf-4c1e-9da6-5175644fe22d
  2. Login to your server as an administrator.
  3. Open Windows Explorer.
  4. Browse to the downloaded .msu file and double click it.  In the case of a x64 server, the file name is “Windows6.1-KB982307-x64.msu”.
  5. Confirm installation by clicking “Yes”. image_7_6893988B
  6. Wait while the updates is being installed.image3_6893988B
  7. Upon completion of the installation, you will need to restart your server.  Click “Restart Now” to complete the installation.image6_6893988B


Cheers
C

24 November 2010

How do I – Use a bitmask enum in C#

In today’s world where processor memory is plentiful, many developers will often times just use a List<bool> for tracking a series of on/off values.  Though the compiler can optimize this code somewhat, the CPU cycles spent managing the List<> structure when super fast bitwise operations could have been done, isn’t really the best way to go.  So when one of my mentees asked about checking values in an enum bitmask, I thought it best to blog the answer for the benefit of others as well.
Let’s assume I’m trying to track the on/off values of say a permissions mask.  I would define my values using the Flags attribute on an enum thus:
[Flags]
public enum Permissions
{
    All = 4,
    Update = 2,
    Read = 1,
    None = 0
}

If I now proceed to define a variable of our enum, I can set some of it’s bit values using the the bitwise AND operator thus:
Permissions perm = Permissions.Read & Permissions.Update;

If I now check for a value in the mask that is NOT turned on, it is done with the bitwise OR operator thus:
if ((perm & Permissions.All) != 0)
{
    //it never gets here
}

The same applies for values that ARE turned on in the mask thus:
if ((perm & Permissions.Update) != 0)
{
    //it gets here
}

In addition, I can turn on more flag altering future check for them thus:
perm = perm & Permissions.All;
if ((perm & Permissions.All) != 0)
{
    //now it gets here
}

The major advantage to using the bitmask enum with bitwise operators is speed.  These operations are done by the processor literally just flipping a single bit.
Happy coding!



Cheers
C

23 November 2010

How do I – Write my own C Sharp IEqualityComparor when System.Collections.Generic.List.Contains(System.Xml.Linq.XNode) fails and does not work as expected

Unfortunately I ran into an issue with the C# List<T> class the other day.  I was in need of checking for a given XML snippet’s existence in an XML document.  I immediately jumped in and tried using Linq2Xml and List<T> to solve the problem.  The way I figured it, I could grab a quick list of XNodes from the XML document and then using the .Contains() method, I could check if the node in the XML snippet exists in the list.  Imagine my surprise when I ran through the code, but for some reason, the .Contains() method did NOT return true even on an identical match.  Weird.  :S
After spending some time trying to figure out why it wasn’t working as advertised (as I’ve mentioned before, 80% of a developer’s time is spent figuring out why something isn’t working as advertised or published) I decided to go the easier route and leverage the .Contains() method’s second overloaded form which takes an IEQualityComparor so all I’d have to do is write my own IEQualityComparor derived class and pass it to the .Contains() method.  Let’s take a look at how we do this.


In this post we’ve shown how to create our very own IEqualityComparor method to use for our purposes since the List<XNode>.Contains() method doesn’t function the way we needed it to.  I would venture to say, that the .Contains() method without our IEqualityComparor is pretty much useless.
Of course, if anyone out there can explain to me why it works the way it does and how the way it works is actually useful, I’m all ears. 🙂


Cheers
C

18 November 2010

Process Explorer 14 now available!



If you don’t know who Mark Russinovich is, you’ve been living under a rock. Even so, if you spend ANY time in the Windows world, you’d want to get Mark’s latest iteration of Process Explorer, now at version 14!

Come and get it! http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx

Cheers
C

17 November 2010

Start thinking in parallel… seriously!

As we move forward with Visual Studio 2010 and the .NET Framework 4.0, it will become more and more important to understand, embrace and implement parallel programming patterns.  Even though SharePoint 2010 still only runs on .NET 3.5, projects and components that doesn’t directly call into SharePoint, can be built upon this framework technology.
With servers now spanning multi proc/multi core (dual proc/hex core for most standard servers today), it means that a traditional single threaded application only leverages 1/24th (12 cores plus hyper-threading) of it’s potential processor power.  Using simple components such as Parallel.For for instance can potentially yield much better performance and throughput to the apps we build.  I’m by no means advocating consuming all 24 cores in your application on the server, but the point is that we should be thinking in parallel mode.
That said, Stephen Toub wrote the nice white paper on the subject.  Yes, it’s 118 pages, but it’s well worth the read, even if just to stimulate peripheral awareness of the parallel shift in coding.  I enjoyed it and I hope you do too! 🙂
The paper is published here:  http://www.microsoft.com/downloads/en/details.aspx?FamilyID=86b3d32b-ad26-4bb8-a3ae-c1637026c3ee&displaylang=en
For my own reference, I’ve also saved it off to my downloads folder just in case the above link isn’t functional some time in the future.
http://www.cjvandyk.com/blog/Downloads/Patterns%20for%20Parallel%20Programming%20-%20Understanding%20and%20Applying%20Parallel%20Patterns%20with%20the%20.NET%20Framework%204.pdf

Cheers
C

10 November 2010

New SharePoint blogger worth following



My friend, Jason Himmelstein, recently setup the SharePointLonghorn blog and I’ve been eagerly waiting for him to start cranking out content. Well the wait is over. He’s launched into the blogosphere with some awesome posts well worth reading. While you’re at it, subscribe to his RSS feed.

Windows with Claims User gets access denied to a site they had access to earlier in the day

Unable to access VMware Workstation Guest after Host Machine crash Random Server hang issues result in a required hard reset SharePoint 2010 Farm Service Account passwords expired?!?!?!? Like I said… great content! Enjoy!

Cheers
C

07 November 2010

How to deal with orphaned SharePoint sites

Whatever causes it, orphaned sites in SharePoint can be a very painful issue to deal with.  One way in which an orphaned site can occur is when a STSADM command it terminated during execution because of a drop of your remote desktop connection.
If you are working with STSADM, be sure to configure your Remote Desktop settings in order to keep disconnected sessions alive.  Failure to do so, WILL cause problems for you in the future.
The toughest orphaned site I’ve ever had to deal with occurred during such a case.  The site was being restored from a STSADM backup.  The server’s Remote Desktop settings were not set to keep the session alive and when a blip in the wireless network cause the connection to be broken, Windows simply killed the session causing the site collection to become orphaned.  This was evident when the restore command was re-attempted:
stsadm -o restore -url http://server/site -filename site.stsadm.bak -overwrite
Which returned this puzzling message:
Another site already exists at http://server/site. Delete this site before attempting to create a new site with the same URL, choose a new URL, or create a new inclusion at the path you originally specified.
This is a puzzling message because… didn’t we specify the “-overwrite” switch?  Why yes we did!  So what is going on here? OK, OK.  So the site is there and the “-overwrite” switch isn’t working.  Let’s just delete the site.  The following command is issued:
stsadm -o deletesite -url http://server/site
Only thing is, the site collection delete statement failed with the following error:
The system cannot find the path specified. (Exception from HRESULT: 0x80070003)
No problem you say.  Just use the -force switch with the deletesite command.  Doing that, the following command is issued:
stsadm -o deletesite -force -siteid 2a9fbc10-4a60-427b-8e6e-e1565d7e5796 -databaseserver sql -databasename Site_Content
Unfortunately, the response is not what you expected.  This is the error returned:
Database “Site_Content” is not associated with this Web application
Of course, the message would make sense if the database was not associated with the web application, but it was.  It is time for the good old orphan identification and handling swich… databaserepair We begin by identifying the orphaned content.  We issue the following command:
stsadm -o databaserepair -url http://server/site -databasename Site_Content
As expected, the command dumps a bunch of orphaned content.  Time to delete the corrupted content so we can restore the site again.  Simply add the -deletecorruption switch thus:
stsadm -o databaserepair -url http://server/site -databasename Site_Content -deletecorruption
Once the corrupted content has been deleted, rerunning the command without the -deletecorruption switch yields a clean bill of health for our database thus:
<OrphanedObjects Count=”0″ />
Unfortunately, retrying the restore still fails by with the same “site exists” error.  This is where I had the idea to detach and re-attach the database to try and clear up the issue.  Two quick STSADM commands later thus:
stsadm -o deletecontentdb -url http://server/site -databasename Site_Content
and
stsadm -o addcontentdb -url http://server/site -databasename Site_Content
yielded:
Operation completed successfully.
Now retrying the restore thus:
stsadm -o restore -url http://server/site -filename site.stsadm.bak -overwrite
finally yielded the result we were looking for:
Operation completed successfully.
  Hopefully this will help you get sticky orphaned sites handled quickly so you can move on to more important things!

Cheers
C

04 November 2010

How do I – Create a Windows Service application using Visual Studio 2010



We don’t build many Windows Service applications any more these days, but every once in a while, the need for one comes around. With Visual Studio 2010 now the mainstay of our tool set, let’s look at how you accomplish the mission using this tool.

http://www.cjvandyk.com/blog/Articles/How-do-I–Create-a-Windows-Service-application-using-Visual-Studio-2010.aspx



Cheers
C

01 November 2010

Why can’t I use SCRIPT or IFRAME tags in my SharePoint Rich Text fields?

I answered this question and thought it might be good to post for others as well…  The question was: “So before I write a custom field to replace the use of the OOTB multiline text field could anyone tell me a couple things? First, could someone better explain the reason that script and iframe tags were allowed in content editor web parts but not in the multiline text RTE? Second, is there a way to enable the use of unsafe tags within the multiline text field without having to create a custom field?”   The answers to your questions are: 1.  SECURITY. 2.  NO. (See 1 above) So now, let me explain… The use of <script> and <iframe> tags in the Rich Text fields are not allowed, or rather, are not interpreted as their types, but just as text, because it’s a rich TEXT field.  As a rich text field, the content of the field is something that a USER can set.  As such, any web site that would allow a USER to set the content of a field to something that is executable such as SCRIPTS or IFRAMES, would pose a grave security risk.  It’s like telling a hacker… Here’s the keys to my server.  Do your worst. For that reason, all fields that contain content set by users, are configured to NOT allow users to embed scripts etc. into the pages. In the same way, the use of Content Editor web parts is limited to users with DESIGNER or ADMINISTRATOR rights.  The assumption here is that these level users are trusted users that have been vetted and they won’t intentionally embed harmful content into pages. SharePoint isn’t trying to make life hard… it’s just protecting us from ourselves sometimes. 🙂

Cheers
C

26 October 2010

How do I – Restore access to SharePoint 2010 Central Admin pages when I get Unexpected Error Occurred

If you’re having issues accessing your SharePoint 2010 Central Admin pages, but SharePoint just gives you the infamous “Unexpected Error” message, this might help.
First thing you want to do is check the Windows Event Log for errors in the w3wp.exe process.  If you see the following error:
06/16/2010 09:30:31.53  w3wp.exe (0x1468)                        0x0534 SharePoint Foundation          Runtime                        Unexpected System.Security.Policy.PolicyException: Required permissions cannot be acquired.    at System.Security.SecurityManager.ResolvePolicy(Evidence evidence, PermissionSet reqdPset, PermissionSet optPset, PermissionSet denyPset, PermissionSet& denied, Boolean checkExecutionPermission)     at System.Security.SecurityManager.ResolvePolicy(Evidence evidence, PermissionSet reqdPset, PermissionSet optPset, PermissionSet denyPset, PermissionSet& denied, Int32& securitySpecialFlags, Boolean checkExecutionPermission) b4efe4b0-ba91-4f03-a888-a952b8372fb4
The error reflected here indicates that the executing process was unable to get the proper execution permissions from the CLR i.e. trust levels aren’t where they need to be.  Time to dig into the  web.config for the Central Admin site.
Look for the “trust level” node in web.  Odds are it’s set to “WSS_Minimal”.  You’d want to change that node to be:
<trust level=”Full” originUrl=”” /> Then simply give IIS a quick kick and retry access to Central Admin.  All should be well now… IMPORTANT NOTE:  USING FULL TRUST AS IN THIS CASE, IS NOT RECOMMENDED AND IS NOT A BEST PRACTICE!  THIS IS SIMPLY A QUICK FIX. For best practices on implementing Code Access Security in SharePoint, please see this awesome article my friend Andrew Connell wrote.

Cheers
C

21 October 2010

How do I – Solve the problem where the MS Word Document Information Panel dropdown does NOT show all the values from the source SharePoint list

I was asked this question last night. “Why doesn’t all my lookup list values show up in the Document Information Panel in Office?”
This one is a tricky little one to try and debug. Unless you know a little about what is going on in the plumbing of SharePoint and how it works behind the scenes, you may spend more time hunting this than you should. Hopefully this post will save you some time. 😉
Back to the problem…
Let’s start with the source list against which the lookup is done. Let’s define a list called “TestDropdownSource”. There’s nothing special about this list. Just add a couple of records to the list.
image_29_3D505C6E
As you can see here, the All Items view for this list is just showing the first 3 entries, even though we have 5 records in the list.
image_30_3D505C6E
If we look at the definition of the view in question, and then scroll all the way down and expand the “Item Limit” section, we see that I defined the number of items to display as 3 for the purpose of this demo.
image_31_3D505C6E
Now we create a new document library called “TestSource”. We add a new column to the library and use a lookup to our source list. The library looks like this:
image_32_3D505C6E
When we look at the Columns, we notice the “LookupValue” field.
image_33_3D505C6E
When we look at the definition of the “LookupValue” field, we notice that it gets its information from the “TestDropdownSource” list’s Title field.
image_34_3D505C6E
If we now add a new document to the document library (which is set to use Word), Word will open and the Document Information Panel will show the “LookupValue” field. When we open the dropdown list, we notice that ONLY 3 items are shown.
image_35_3D505C6E
That is the problem that we are trying to solve. To fix this, we go back to the list definition for the “TestDropdownSource” list.
We open the default view’s definition. In this case, we only have the “All Items” view, but if you have more views defined, you are interested in the default view. That is because Word’s implementation of the SharePoint web services uses the default view of the source list rather than a targeted view.
Once in the view’s definition, we are going to set the number of items back up to 100.
image_36_3D505C6E
A quick check and we see all 5 items displayed in the view.
image_37_3D505C6E
Now simply create a new document again (or open an existing one).
image_38_3D505C6E
And check the DIP… all should be well… 🙂
image_39_3D505C6E

Cheers
C

20 October 2010

How do I – Filter values in two lists of custom classes using Linq

I originally titled this post “Linq – So wonderful and oh so damn frustrating”, but then decided that the post probably warrants its current name since someone might actually be trying to do the same thing and search for examples of how to achieve that.  There are tons of examples of just how cool Linq is and I’m not saying that it isn’t, but man it can be so frustrating to work with at times.  Now I’m not going to pretend that I’m a Linq expert.  I’m not.  I know just enough to be dangerous.  Linq has been useful to me in the past, but this week I stumbled across an issue that just drove me batty!
<rant>
Just this weekend I was telling Jess (my wonderful SciFi, geeky wife) that 80% of a developer’s time is spent figuring out why code (methods & APIs) does NOT work the way it’s supposed to.  Not how we THINK it’s supposed to work, but how it was PUBLISHED and advertised to work. 
</rant>
OK, off my soapbox and back to the Linq issue…
I have a class defined as SystemFile.  I’m basically trying to compare a folder with all its files and sub folders to another folder with its files and sub folders.  In the process I have two lists of SystemFile containing the info about the two folders I wish to compare.  Using Linq to compare the lists, we can define a comparer class of type IEqualityComparer<T> to do the comparison of our custom class and assist in the filtering.  My comparer class is defined thus:

public class SystemFileGACComparor : IEqualityComparer<SystemFile>
{
    public bool Equals(SystemFile source, SystemFile target)
    {
        if (source.FullPath.ToLower().Contains(@"c:\windows\assembly\temp"))
        {
            return true;
        }
        else
        {
            if (source.FullPath.ToLower().Contains(@"c:\windows\assembly\tmp"))
            {
                return true;
            }
            else
            {
                if (source.FullPath == target.FullPath)
                {
                    return true;
                }
                else
                {
                    return false;
                }
            }
        }
    }

    public int GetHashCode(SystemFile source)
    {
        return source.FullPath.GetHashCode();
    }
}

A note about my logic.  The easiest code implementation of the Equals() method could have been simply as
   return (source.FullPath == target.FullPath);
but since I am actually comparing GAC folders in this case, there are the the “Temp” and “Tmp” folders to consider (and exclude) in my comparison.  Since “Tmp” is used during installation and “Temp” during uninstallation of of assemblies, they will have temporary GUID values in their path names which will always be different between systems.  As a result, we have to exclude anything from these folders in our comparison.  For that reason, I added the checks for these folders in the path of the source being checked.
With the comparer class in place, let’s implement it.  The code is straight forward thus:

SystemFileGACComparor compare = new SystemFileGACComparor();
IEnumerable<SystemFile> sfSource = LoadFileListFromXML("1WEB.xml");
IEnumerable<SystemFile> sfTarget = LoadFileListFromXML("1APP.xml");
List<SystemFile> lstOnSourceNotTarget = sfSource.Except(sfTarget, compare).ToList();
First we define a an instance of the comparer class for use.
Then we load the XML dump of our two lists into an IEnumerable<SystemFile> structure so that Linq can work on them.
Now simply ask Linq to compare the two lists and produce a list with the differences.  Per the published MSDN documentation, the Except() method will take our sfSource and remove any records that it finds in sfTarget that match, finally returning what remains.
For the record, here is what I’m comparing:
1WEB.xml
<?xml version=”1.0″ encoding=”utf-8″?>
<GAC>
  <File FullPath=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a\adodb.dll” FileName=”adodb.dll” Size=”110592″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a” CreatedAt=”2010-07-07T14:27:58.103-04:00″ LastModifiedAt=”2010-07-07T14:27:58.119-04:00″ LastAccessedAt=”2010-07-07T14:27:58.103-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a\__AssemblyInfo__.ini” FileName=”__AssemblyInfo__.ini” Size=”196″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a” CreatedAt=”2010-07-07T14:29:40.671-04:00″ LastModifiedAt=”2010-07-07T14:29:40.703-04:00″ LastAccessedAt=”2010-07-07T14:29:40.671-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a” FileName=”7.0.3300.0__b03f5f7f11d50a3a” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\GAC\ADODB” CreatedAt=”2010-07-07T14:27:58.103-04:00″ LastModifiedAt=”2010-07-07T14:29:40.671-04:00″ LastAccessedAt=”2010-07-07T14:29:40.671-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\ADODB” FileName=”ADODB” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\GAC” CreatedAt=”2010-07-07T14:29:40.703-04:00″ LastModifiedAt=”2010-07-07T14:29:40.718-04:00″ LastAccessedAt=”2010-07-07T14:29:40.718-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.0__b03f5f7f11d50a3a\envdte.dll” FileName=”envdte.dll” Size=”245760″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.0__b03f5f7f11d50a3a” CreatedAt=”2010-07-13T11:42:59.953-04:00″ LastModifiedAt=”2010-07-13T11:42:59.968-04:00″ LastAccessedAt=”2010-07-13T11:42:59.953-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.0__b03f5f7f11d50a3a\__AssemblyInfo__.ini” FileName=”__AssemblyInfo__.ini” Size=”194″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.0__b03f5f7f11d50a3a” CreatedAt=”2010-07-13T11:43:16.625-04:00″ LastModifiedAt=”2010-07-13T11:43:16.625-04:00″ LastAccessedAt=”2010-07-13T11:43:16.625-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.0__b03f5f7f11d50a3a” FileName=”8.0.0.0__b03f5f7f11d50a3a” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\GAC\EnvDTE” CreatedAt=”2010-07-13T11:42:59.953-04:00″ LastModifiedAt=”2010-07-13T11:43:16.625-04:00″ LastAccessedAt=”2010-07-13T11:43:16.625-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\EnvDTE” FileName=”EnvDTE” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\GAC” CreatedAt=”2010-07-13T11:43:16.64-04:00″ LastModifiedAt=”2010-07-13T11:43:16.64-04:00″ LastAccessedAt=”2010-07-13T11:43:16.64-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp\30ROE94YXW\One.MasterPages.dll” FileName=”One.MasterPages.dll” Size=”6144″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\temp\30ROE94YXW” CreatedAt=”2010-10-11T10:43:21.996-04:00″ LastModifiedAt=”2010-10-11T10:43:21.996-04:00″ LastAccessedAt=”2010-10-11T10:43:21.996-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp\30ROE94YXW” FileName=”30ROE94YXW” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\temp” CreatedAt=”2010-10-11T15:27:36.922-04:00″ LastModifiedAt=”2010-10-11T15:27:36.922-04:00″ LastAccessedAt=”2010-10-11T15:27:36.922-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp\5ZY4HSUIYK\One.EVMS.Dashboard.dll” FileName=”One.EVMS.Dashboard.dll” Size=”837632″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\temp\5ZY4HSUIYK” CreatedAt=”2010-10-04T08:56:07.183-04:00″ LastModifiedAt=”2010-10-04T08:56:07.277-04:00″ LastAccessedAt=”2010-10-04T08:56:07.183-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp\5ZY4HSUIYK” FileName=”5ZY4HSUIYK” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\temp” CreatedAt=”2010-10-11T10:51:38.965-04:00″ LastModifiedAt=”2010-10-11T10:51:38.965-04:00″ LastAccessedAt=”2010-10-11T10:51:38.965-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp” FileName=”temp” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly” CreatedAt=”2009-07-14T00:58:28.892-04:00″ LastModifiedAt=”2010-10-11T17:33:53.016-04:00″ LastAccessedAt=”2010-10-11T17:33:53.016-04:00″ />
  <File FullPath=”C:\Windows\Assembly\tmp” FileName=”tmp” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly” CreatedAt=”2010-07-07T14:18:45.924-04:00″ LastModifiedAt=”2010-10-11T17:35:50.969-04:00″ LastAccessedAt=”2010-10-11T17:35:50.953-04:00″ />
</GAC>
1APP.xml
<?xml version=”1.0″ encoding=”utf-8″?>
<GAC>
  <File FullPath=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a\adodb.dll” FileName=”adodb.dll” Size=”110599″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a” CreatedAt=”2010-07-07T14:27:58.103-04:00″ LastModifiedAt=”2010-07-07T14:27:58.119-04:00″ LastAccessedAt=”2010-07-07T14:27:58.103-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a\__AssemblyInfo__.ini” FileName=”__AssemblyInfo__.ini” Size=”196″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a” CreatedAt=”2010-07-07T14:29:40.671-04:00″ LastModifiedAt=”2010-07-07T14:29:40.703-04:00″ LastAccessedAt=”2010-07-07T14:29:40.671-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\ADODB\7.0.3300.0__b03f5f7f11d50a3a” FileName=”7.0.3300.0__b03f5f7f11d50a3a” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\GAC\ADODB” CreatedAt=”2010-07-07T14:27:58.103-04:00″ LastModifiedAt=”2010-07-07T14:29:40.671-04:00″ LastAccessedAt=”2010-07-07T14:29:40.671-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\ADODB” FileName=”ADODB” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\GAC” CreatedAt=”2010-07-07T14:29:40.703-04:00″ LastModifiedAt=”2010-07-07T14:29:40.718-04:00″ LastAccessedAt=”2010-07-07T14:29:40.718-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.1__b03f5f7f11d50a3a\envdte.dll” FileName=”envdte.dll” Size=”245760″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.0__b03f5f7f11d50a3a” CreatedAt=”2010-07-13T11:42:59.953-04:00″ LastModifiedAt=”2010-07-13T11:42:59.968-04:00″ LastAccessedAt=”2010-07-13T11:42:59.953-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.1__b03f5f7f11d50a3a\__AssemblyInfo__.ini” FileName=”__AssemblyInfo__.ini” Size=”194″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.0__b03f5f7f11d50a3a” CreatedAt=”2010-07-13T11:43:16.625-04:00″ LastModifiedAt=”2010-07-13T11:43:16.625-04:00″ LastAccessedAt=”2010-07-13T11:43:16.625-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\EnvDTE\8.0.0.1__b03f5f7f11d50a3a” FileName=”8.0.0.0__b03f5f7f11d50a3a” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\GAC\EnvDTE” CreatedAt=”2010-07-13T11:42:59.953-04:00″ LastModifiedAt=”2010-07-13T11:43:16.625-04:00″ LastAccessedAt=”2010-07-13T11:43:16.625-04:00″ />
  <File FullPath=”C:\Windows\Assembly\GAC\EnvDTE” FileName=”EnvDTE” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\GAC” CreatedAt=”2010-07-13T11:43:16.64-04:00″ LastModifiedAt=”2010-07-13T11:43:16.64-04:00″ LastAccessedAt=”2010-07-13T11:43:16.64-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp\33ROE94YXW\One.MasterPages.dll” FileName=”One.MasterPages.dll” Size=”6144″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\temp\30ROE94YXW” CreatedAt=”2010-10-11T10:43:21.996-04:00″ LastModifiedAt=”2010-10-11T10:43:21.996-04:00″ LastAccessedAt=”2010-10-11T10:43:21.996-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp\33ROE94YXW” FileName=”30ROE94YXW” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\temp” CreatedAt=”2010-10-11T15:27:36.922-04:00″ LastModifiedAt=”2010-10-11T15:27:36.922-04:00″ LastAccessedAt=”2010-10-11T15:27:36.922-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp\53Y4HSUIYK\One.EVMS.Dashboard.dll” FileName=”One.EVMS.Dashboard.dll” Size=”837632″ IsReadOnly=”false” IsFolder=”false” DirectoryName=”C:\Windows\Assembly\temp\5ZY4HSUIYK” CreatedAt=”2010-10-04T08:56:07.183-04:00″ LastModifiedAt=”2010-10-04T08:56:07.277-04:00″ LastAccessedAt=”2010-10-04T08:56:07.183-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp\53Y4HSUIYK” FileName=”5ZY4HSUIYK” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly\temp” CreatedAt=”2010-10-11T10:51:38.965-04:00″ LastModifiedAt=”2010-10-11T10:51:38.965-04:00″ LastAccessedAt=”2010-10-11T10:51:38.965-04:00″ />
  <File FullPath=”C:\Windows\Assembly\temp” FileName=”temp” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly” CreatedAt=”2009-07-14T00:58:28.892-04:00″ LastModifiedAt=”2010-10-11T17:33:53.016-04:00″ LastAccessedAt=”2010-10-11T17:33:53.016-04:00″ />
  <File FullPath=”C:\Windows\Assembly\tmp” FileName=”tmp” Size=”0″ IsReadOnly=”false” IsFolder=”true” DirectoryName=”C:\Windows\Assembly” CreatedAt=”2010-07-07T14:18:45.924-04:00″ LastModifiedAt=”2010-10-11T17:35:50.969-04:00″ LastAccessedAt=”2010-10-11T17:35:50.953-04:00″ />
</GAC>
When we run through the code and break after the Except() method, this is what we see for the sfSource an sfTarget:

image
As we expected, we see all the files in the sfSource.  Now let’s look at the value of the results list lstOnSourceNotTarget:

image
Hmm… that’s curious… I expected the three EnvDTE records to be there, but the \temp\ files SHOULD have been filtered out by our comparer class’ Equal() method, right?
Confused, I set a break inside our comparer class and rerun our code to see what is actually being compared and filtered.  This is what we see:
Breaking here:

image
First break

image
Second break

image
Third break

image
Fourth break

image
Fifth break

image
Sixth break

image
Seventh break

image
And ???

image
Hmm… Seven breaks for 14 files and NONE of them were the 4 that shows up at the end with \temp\ in the name.  WTF???!!!
Are you confused?  I sure am!!!
The closest thing to a “rationalization” I can make for myself on this is that it has something to do with Linq’s LAZY nature.  So the items doesn’t get checked unless I iterate over them.  (I thought that’s what the Except() method was doing, but oh well…).
OK, enough time spent on something that DOESN’T WORK AS PUBLISHED!!!
Let’s get a workaround in place…
We can use the Where() method in Linq to get the subset of records that actually contains the string we’re trying to parse out and then reversing our logic, we can pass that set of records to the Except() method to exclude them from the original list.  Our new code looks like this: 

SystemFileGACComparor compare = new SystemFileGACComparor();
IEnumerable<SystemFile> sfSource = LoadFileListFromXML("1WEB.xml");
sfSource = sfSource.Except(
    sfSource.Where(filter => filter.FullPath.ToLower().Contains(@"c:\windows\assembly\temp")), compare).Except(
    sfSource.Where(filter => filter.FullPath.ToLower().Contains(@"c:\windows\assembly\tmp")), compare);
IEnumerable<SystemFile> sfTarget = LoadFileListFromXML("1APP.xml");
sfTarget = sfTarget.Except(
    sfTarget.Where(filter => filter.FullPath.ToLower().Contains(@"c:\windows\assembly\temp")), compare).Except(
    sfTarget.Where(filter => filter.FullPath.ToLower().Contains(@"c:\windows\assembly\tmp")), compare);
List<SystemFile> lstOnSourceNotTarget = sfSource.Except(sfTarget, compare).ToList();

We start by taking the list and applying the Where() method against it.  Inside the Where() method expression, we use the .FullPath property and convert its value to all lower case using the ToLower() method.  Once in all lower case, we use the Contains() method to check for the “temp” reference.  This will produce a list of only the records that actually contains the “temp” values.  Passing that off to the Except() method leaves us with the original list MINUS the records containing “temp”.
Wash, rinse, repeat…
We simply drop in a second Except() with the same code and a reference to “tmp” instead and tada!  We have a list that doesn’t contain either “temp” or “tmp”.
Finally, we can move onto the next problems that doesn’t work as published…



Cheers
C

SharePoint Remote Event Receivers are DEAD!!!

 Well, the time has finally come.  It was evident when Microsoft started pushing everyone to WebHooks, but this FAQ and related announcement...