When Using SharePoint, Don’t Use $

Hey folks!

I ran into an interesting issue while onsite with a customer the other week and I thought maybe someone else could benefit from this. The scenario had to do with the Distributed Cache and we were trying to correct some issues that they had.

The basic steps were to Remove/Add the SPDistributedCacheServiceInstance using powershell (PS) in order to ‘fix’ it. When we executed the Add-* PS cmdlet we ended up seeing the following error:


This was most vexing as the user running the PS cmdlet was a Farm Admin and local admin on the server in question. I consulted with several of my peers when it occurred to one of them that the referenced path appeared quite interesting:


The actual user account was not test18172account, but was test$account. Now if you have done your homework, there is a specific supportability statement with regards to SharePoint service accounts:

Do not use service account names that contain the symbol $.

So there is a small bit of room for discussion about this since the article states “service account names” and in this scenario we weren’t using a service account. However, as it turns out this restriction apparently applies to accounts used running certain PS cmdlets.

As soon as we moved to another admin account that didn’t contain a $ everything worked as expected. So the morale of the story is to avoid using special characters in your account names — or at least $ (dollar signs).  🙂

Reposting the error in text for search-ability:

Add-SPDistributedCacheServiceInstance : Could not find a part of the path
At line:1 char:1
+ Add-SPDistributedCacheServiceInstance
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Share…ServiceInstance:SPCmdletAddDist…ServiceInstance) [Add-
   SPDistributedCacheServiceInstance], CmdletInvocationException
    + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletAddDistributedCacheServiceInstance


Changing the Service Bus RunAs Account Password Changes VSS Credentials

Since I’ve been working in the area of Workflow Manager (WFM) as it relates to SharePoint and since WFM is ‘tied at the hip’, so to speak, to Service Bus one of my co-workers forwarded me some interesting information regarding Service Bus that I thought was worth letting you know about if if you didn’t already…

<wow, that was a long and I’m sure grammatically incorrect sentence>

Basically, my peer, Joe Rodgers, is a powershell nut and uses it for everything. He was writing some scripts for one of his customers and noticed that when he changes the password for his Service Bus RunAs account, that the credentials for the Service Bus VSS service also change to match that RunAs account. Initially when you configure SB along with WFM for SharePoint, the Service Bus VSS service is configured to use LocalSystem.

Before the change:


The script used:


After the change:


Notice that the script doesn’t mention the account name, but what happens is in the code for setting the password of the SB RunAs account it also takes those credentials and sets them on the Service Bus VSS service.

I don’t expect this to affect many of us, but it’s possible that this will affect you if you are using some product that hooks into the VSS writers.

401 – Unauthorized Response to Register-SPWorkflowService

My previous post about configuring the Workflow Manager using least privilege is a great start; however I realized the other day that I was missing something. Thinking about a more realistic scenario and indeed if I decided that I need high availability, then I’d need more than one Workflow Manager (WF) server.

I started doing research and found a great 3 or 4 part series of blogs from the well-known Spencer Harbar. In this blog series, Spencer goes through a great explanation of the certificates involved and walks through creating a highly available WF Farm using Windows Load Balancer (WLB) software. His explanation describes how the Subject Alternative Name (SAN) field is used to allow us to use virtual names easily.

I’m not going to re-hash the setup of WF in this post, but I did alter the setup in order to use SSL. My configuration for this post includes:

  • a fully configured SharePoint Server 2013 Farm with a host named site collection at https://intranet.contoso.lab
  • a WF Farm consisting of a single server and configured for HTTPS, but I want to register it within SharePoint as a virtual name in order to ease expansion in the future:

    Server: WF01.contoso.lab

    Virtual name: WF.Contoso.lab

So we’ve created a DNS A record for the virtual name of wf.contoso.lab and it’s resolving to the IP address of the server wf01.contoso.lab. Next we run the powershell cmdlet to register the WF Farm with the SharePoint Farm and we expect all of this to work without a problem… because nothing ever fails the first time around, right? Riiiiiiiggghhhtt…

So we log into the SharePoint server as an appropriate account (refer to my previous post), and open a SharePoint 2013 Management Shell and execute the following:

Register-SPWorkflowService –SPSite https://intranet.contoso.lab –WorkflowHostUri https://wf.contoso.lab:12290

The result?

Register-SPWorkflowService : A response was returned that did not come from
the Workflow Manager. Status code = 401:
< !DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Strict//EN”
< html xmlns=”http://www.w3.org/1999/xhtml”>
< head>
< meta http-equiv=”Content-Type” content=”text/html; charset=iso-8859-1″/>
< title>401 – Unauthorized: Access is denied due to invalid credentials.</title>
< style type=”text/css”>
< !–
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica,
fieldset{padding:0 15px 10px 15px;}
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:”trebuchet
MS”, Verdana, sans-serif;color:#FFF;
#content{margin:0 0 0 2%;position:relative;}
< /style>
< /head>
< body>
< div id=”header”><h1>Server Error</h1></div>
< div id=”content”>
< div class=”content-container”><fieldset>
<h2>401 – Unauthorized: Access is denied due to invalid credentials.</h2>
<h3>You do not have permission to view this directory or page using the
credentials that you supplied
< /fieldset></div>
< /div>
< /body>
< /html>
HTTP headers received from the server – WWW-Authenticate: Negotiate,NTLM.
Client ActivityId : e27eaa36-7e61-4511-a767-7cc5bad42e2e.
At line:1 char:1
+ Register-SPWorkflowService -SPSite https://intranet.contoso.lab

-WorkflowHostUri …
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidData: (Microsoft.Share…WorkflowService:
RegisterSPWorkflowService) [Register-SPWorkflowService], AuthenticationExc
+ FullyQualifiedErrorId : Microsoft.SharePoint.WorkflowServices.PowerShell

So basically, even though I was using all the right accounts in my least privileged configuration — you read my previous post, right? — I was still encountering an access denied. To save you the time and effort of all of the trouble-shooting I will cut to the chase. The following error showed up in the System Event log on my WF server:

Log Name:      System
Source:        Microsoft-Windows-Security-Kerberos
Event ID:      4
Level:         Error
User:          N/A
Computer:      wf01.contoso.lab

The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server wf01$. The target name used was HTTP/wf.contoso.lab. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Ensure that the target SPN is only registered on the account used by the server. This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service. Ensure that the service on the server and the KDC are both configured to use the same password. If the server name is not fully qualified, and the target domain (CONTOSO.LAB) is different from the client domain (CONTOSO.LAB), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.

After consulting with a few peers, they immediately suspected kernel mode auth in IIS. As a test, we disabled that option on the WF website and reset IIS (to refresh Kerberos tickets) and now when registering the SPWorkflowService in SharePoint it worked flawlessly. However, there are advantages to using kernel mode authentication in IIS 7.0 so we don’t necessarily want to disable it completely.

The explanation of what’s happening…

With kernel mode authentication enabled, IIS utilizes http.sys to decrypt Kerberos tickets and it will be doing so under the context of the System Account (notice the reference in the event log error above to ‘wf01$’). When this happens with an application pool that is running under specific account credentials, the Kerberos tickets cannot be decrypted and used.

Instead of disabling kernel mode authentication, we have another option to allow the application pool credentials to be used to decrypt Kerberos tickets – useAppPoolCredentials. The command for this is as follows:

appcmd.exe set config “Workflow Management Site” –section:system.webServer/security/authentication/windowsAuthentication /useAppPoolCredentials:”True” /commit:apphost

After running this, with kernel mode authentication enabled on the Workflow Management Site, then the registration cmdlet should succeed. Note that if you have run the registration command previously and it failed, then you may need to add the –Force parameter to it in order to overwrite the previously failed registration.

As with the majority of the content I produce, I could not have done this alone (or at least not as quickly). Kudos out to a couple of peers:

  • Dean Cron (fellow PFE)
  • Xuehong Gan
  • Vishal Bajal

Export-SPWeb Syntax Changes Between Root Site and Sub Sites

Working on a somewhat related issue today and needed to test something with Export-SPWeb. We were trying to import a list from an STP (List template) and in order to get there I decided to export one of my own lists and play around a bit. Much to my surprise I was unable to export a custom list from my lab environment. I kept getting the following error:

export-spweb : The URL provided is invalid. Only valid URLs that are site collections or sites are allowed to be exported using stsadm.exe.
At line:1 char:1
+ export-spweb $web -Path exported_list.cmp -ItemUrl “/lists/testlist” -Force +
+ CategoryInfo : InvalidData: (Microsoft.Share…CmdletExportWeb: SPCmdletExportWeb) [Export-SPWeb], SPException
+ FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletExportWeb

It was quite maddening as the parameters are not that difficult to figure out. (get-help export-spweb)

So I was trying many things and discovered that in a second lab environment I had that it worked flawlessly and I was determined to discover why. The first thing that a peer of mine found for me was that in the failing environment the web from which I was trying to export the custom list was on a site collection under the ‘sites’ managed path (in other words not the root site). The environment that was working was the root site collection.

To cut to the chase, you may not be able to use the same syntax depending on where the list resides. The parameter in question that kept failing for me was the ItemUrl parameter. It is to represent the url to the list that you would like to export. Basically if the site collection is the root site collection, then the ItemUrl should contain a leading “/” such as this:

Export-SPWeb http://intranet.contoso.lab –Path exported_list.cmp –ItemUrl “/Lists/TestList”

However if the site collection is not the root, then you should either include the full server relative url to the list or the web relative url to the list *without* the leading slash:

Export-SPWeb http://intranet.contoso.lab/sites/TestImport –Path exported_list.cmp –ItemUrl “/sites/TestImport/Lists/TestList”


Export-SPWeb http://intranet.contoso.lab/sites/TestImport –Pat exported_list.cmp –ItemUrl “Lists/TestList”

If you care to know, basically under the covers the Export-SPWeb cmdlet will call SPWeb.GetList(), but prior to calling that function it will prepend the SPWeb.ServerRelativeUrl and an extra “/” if the string in ItemUrl does not have a leading “/”. In short, to be safe if you always set the ItemUrl parameter to the server relative url and then path to the list, it should always work regardless of web location.

Powershell for Super Bowl

I’m sure this has been done before and likely in a more elegant fashion, but I was sitting at my desk preparing for my Super Bowl gathering of friends and wanted a different way to select the numbers for the squares than just picking numbers from a hat. Now don’t get me wrong… hats work great especially for a small set of numbers like we are discussing here (0 – 9)… twice… but since I was sitting at my desk and I’m trying to be more dev minded in my daily tasks I thought I’d give Powershell a shot.

I knew they had added a cmdlet Get-Random to Powershell and that should make it easy, but I also needed to add code to make sure we didn’t ‘randomly’ re-select any numbers. Easy enough using a array… so here’s the final product and please keep criticism to a minimum as I’m the stereo-typical IT person that enjoys writing code, but I am far from a developer. J I do invite suggestions for making it better/different as I firmly believe the more ways you know how to accomplish a task the better off you’ll be.

# Originally written by Brian Gregor for the purpose of filling out a 
# Super Bowl squares sheet of 0-9. The get-random cmdlet will not 
# select the max value so for 0-9 you must enter 0 as min and 10 as max

# array to represent numbers already selected
$returned = @()

$min = Read-Host 'Minimum number in range'
$max = Read-Host 'Maximum number in range'

Read-Host 'Press <enter> to retrieve first random number'

# Loop and continue to get numbers checking that they have not
# been retrieved previously
    $number = Get-Random -Minimum $min -Maximum $max
    if(($returned -eq 0) -or (!$returned.Contains($number)))
        Write-Host $number
        $returned = $returned + $number        

        # if all numbers between min/max have been retrieved, then set exit $response
        if($returned.Count -ne ($max - $min))
            $response = Read-Host "Retrieve another number? (Y/N)"        
            $response = "N"
until($response.ToUpper() -eq "N")

Write-Host 'Numbers retrieved: '
foreach($number in $returned)
    Write-Host $number

This script can be downloaded from here.

Getting a Lync Attendee List When Lync Web Access is the Client

When holding Lync meetings with customers who cannot join our meetings through the Lync client they are forced to use the Lync Web Access client, which provides them with a browser based client to join and participate when they otherwise would not be able to do so due to corporate security requirements.

While this works well most of the time there are limitations. I’m not a Lync engineer so I won’t go into other challenges, but this post is to address a single limitation regarding the attendee list. I discovered a while back that OneNote and Lync integrate to the extent that you can get an auto-populated attendee list. The challenge is that when the attendees use LWA, they only show in OneNote as “Guest” as shown below.

OneNote_Lync Integration

So when I need a list of attendees, it works great if everyone is using the full Lync client. When they use LWA, it requires a little more work to get the list. This is where saving conversation history in Outlook really helps and some Powershell.


Open the item in Outlook and copy the TO: line.

**NOTE: I have noticed that sometimes you actually get user names here in addition to ‘guest’, but sometimes not. I’m not sure what determines that, but will update if I discover the reasons. If all you see are ‘guest’ entries, then this process will not work as intended.**

Paste this into Notepad (or other text editor and save as rawdata.txt). This will provide you with a semi-colon delimited list of contacts.

Grab the items and pull them into a Powershell variable and then split them into an array of strings

# Read entire list as a single string from the file
$contacts = Get-Content .\rawdata.txt

# Split into an array of strings on the ‘;’ character
$contacts = $contacts.Split(“;”)

Now, you may notice that there are some duplicates due to the way that attendees enter a lync meeting via LWA and there are entries that you obviously do not want such as “Guest” and phone numbers. So we need to parse those out.

# Remove duplicates
$contacts = $contacts | Select –Unique

# Parse out unwanted entries
$newContacts = @()
$contacts | %{ if(($_.ToLower().Contains("guest")) -or ($_.Contains("+1"))){ write-host 'do nothing with' $_} else { write-host 'adding ' $_ ' to new array'; $newContacts = $newContacts + $_}}

You will also notice that saving this conversation history to Outlook has also attempted to resolve each of the remaining names to your local Global Address List (GAL). So in my case everyone shows up as JoeUser@Microsoft.com. This is obviously not ideal when you are working with contacts outside of your organization so we need to remove that information as well.

# Remove incorrect email domains
$newContacts2 = @()
$newContacts | %{$newContacts2 = $newContacts2 + $_.SubString(0, $_.IndexOf("<"))}

What this should leave you is an array of names that you can then put into your meeting notes or send in an email update regarding the meeting attendees.

Now, obviously you shouldn’t need this much automation if your meeting only had a few attendees, but I’ve been hosting some rather large presentations lately (20 – 80 attendees) and manually massaging the attendee list is tedious at best. Hope this helps!