About Me

My photo
This is a blog for John Weber. One of my joys in life is helping others get ahead in life. Content here will be focused on that from this date forward. John was a Skype for Business MVP (2015-2018) - before that, a Lync Server MVP (2010-2014). I used to write a variety of articles (https://tsoorad.blogspot.com) on technical issues with a smattering of other interests. I have a variety of certifications dating back to Novell CNE and working up through the Microsoft MCP stack to MCITP multiple times. FWIW, I am on my third career - ex-USMC, retired US Army. I have a fancy MBA. The opinions expressed on this blog are mine and mine alone.

2012/05/23

Click to Dial explained

VoipNorm just posted a very nice article on “ Lync 2010 Mobile and Desktop Click-to-Join for Non-Enterprise Voice Users .“  In reading through it, I noticed that he makes the statement that “I have heard people say repeatedly that you need to enable EV for Click-to-Join, but this is not true for either the desktop or mobile clients.”

This statement is accurate in that the user does not need to be EV-enabled, but based on several experiences, I will say that there needs to be some connection to the PSTN for Click-to-Join to work with any client.  I am pointing this out because I have had to explain to a potential client that there needed to be some connection {via gateway, SIP trunk (or some other magic)}, before dial-in conferencing (or outcalling) would work.

Just a clarification from my pea-brain, otherwise, another excellent piece of work from VoipNorm!

YMMV.

2012/05/04

Exchange 2010 refuses to install

Background Information

Recently, I was faced with servers that failed to install with an esoteric error regarding being unable to find a specific GUID and the identity being null. I ended up punting this issue to CSS rather than burn hours repeatedly trying (and failing) to find a solution on my own.  I was happy to discover that it took no less than 5 premier support engineers over 17 hours to resolve this issue – at least I have the satisfaction of knowing that the root cause was bizarre.  Interesting to note that this fix also resolved another issue where autodiscover created a POP account for a non-domain machine but a regular Exchange account for domain machines.
During the failed installs, the local machine would get the Exchange Management tools, but they would not open local, but they would attach at the EMS level to another machine; the EMC would open but then error out on assigning rights for activity.  Usually when you see this type of brew-up, re-running setup /prepareAD puts things back to normal.  However, this action failed for me also.  After the org group fixes, you will note that the EMS fix was to move the IIS to “all unassigned” from a fixed IP.  I believe that I may have set the fixed IP in an effort to get the EMS winRM to work.  If so, my mistake for not moving that back.
Long story short, somebody mail-enabled the Microsoft Exchange Security Groups.  So the GUID that was being looked for by the installer was the Organization Management group.  Once that was fixed, the next group in line caused the same error.  And so on down the line.  The issue trace notes are below.  There are some interesting resolution steps there.

 

The Issue

An Exchange 2010 environment  needed a new CAS – we were installing a new CAS specifically to handle a high number of mobile devices because of CPU thrashing and iPhones.  What should have taken about an hour total ended up in two evenings of my time (and the internal staff time) along with over 17 hours of Microsoft CSS time. 
Exchange 2010 refused to install onto the new server no matter what I did.  Very stubborn.  The Management tools install would go just fine (but then not open properly). 
We got this error in both the GUI install and command line install, as well as the eventlog. 
Description: Exchange Server component Client Access Role failed.
Error: Error: The following error was generated when "$error.Clear();
          if ($RoleIsDatacenter -eq $false)
          {
            $delegatedRoleGroupGuid = [Microsoft.Exchange.Data.Directory.Management.RoleGroup]::DelegatedSetupWkGuid;
            $delegatedSetupRG = Get-RoleGroup $delegatedRoleGroupGuid;
            add-ExchangeAdministrator -role ServerAdmin -Identity $delegatedSetupRG.Identity -Scope $RoleNetBIOSName;
          }
        " was run: "The operation couldn't be performed because object '261928d1-f5d1-445b-866d-1d6b5bd87a09' couldn't be found on 'TACOMA.corp.domain.com'.".
The operation couldn't be performed because object '261928d1-f5d1-445b-866d-1d6b5bd87a09' couldn't be found on 'TACOMA.corp.domain.com'.
Error:  The following error was generated when "$error.Clear();
          if ($RoleIsDatacenter -eq $false)
          { 
$delegatedRoleGroupGuid = [Microsoft.Exchange.Data.Directory.Management.RoleGroup]::DelegatedSetupWkGuid;
            $delegatedSetupRG = Get-RoleGroup $delegatedRoleGroupGuid;
            add-ExchangeAdministrator -role ServerAdmin -Identity $delegatedSetupRG.Identity -Scope $RoleNetBIOSName;
          }
        " was run: "Cannot bind argument to parameter 'Identity' because it is null.".
Cannot bind argument to parameter 'Identity' because it is null.

















Attempting to install the HT role resulted in a version of this error as well.  Except that this one mentioned not being able to find an SMTP connector to the internet  - which was bogus, a connector existed.
Setup cannot detect an SMTP or Send connector with an address space of '*'. Mail flow to the Internet may not work properly.
The EMC would open, and then pitch and error about a remote forest.
clip_image002
The EMS would open, but not connect to the local server but would attach to a remote server.
image
All of these things pointed to a permissions issue; however, adding the role (as a test) to a working mailbox server went just fine.  Really odd.  I also tried setup /prepareAD thinking that “something” went sideways.  No dice.  Setup /prepareAD failed to run.  It just dumped out.  At this point I recommended the client open a case with CSS.

The Solution

First, what happened is that someone (NOT ME!) mail-enabled all the groups in the Microsoft Exchange Security Groups container.  I have never had a reason to do this before; evidently, this is not a recommended action – serious side effects!  Also remember that it took 17 hours for CSS engineers (five of them working in shifts) to resolve this issue.  This work is not for the faint of heart and I would recommend that if you are facing this issue, or think you are, that you call CSS.
The fix (as provided by CSS) is outlined below.
YMMV

-    Setup /prepareAD is failing.
-    Error message received is as below.
[04/27/2012 02:38:43.0455] [2] Used domain controller DC.corp.Company.com to read object CN=Exchange Servers,OU=Microsoft Exchange Security Groups,DC=corp,DC=Company,DC=com.
[04/27/2012 02:38:43.0471] [2] [ERROR] Unexpected Error
[04/27/2012 02:38:43.0471] [2] [ERROR] The well-known object entry with the GUID "6c01d2a7-f083-4503-8132-789eeb127b84", which is on the "CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=corp,DC=Company,DC=com" container object's otherWellKnownObjects attribute, refers to a group "CN=Exchange Servers,OU=Microsoft Exchange Security Groups,DC=corp,DC=Company,DC=com" of the wrong group type. Either delete the well-known object entry, or promote the target object to "Universal, SecurityEnabled".
[04/27/2012 02:38:43.0471] [2] Ending processing initialize-ExchangeUniversalGroups
[04/27/2012 02:38:43.0487] [1] The following 1 error(s) occurred during task execution:
[04/27/2012 02:38:43.0487] [1] 0.  ErrorRecord: The well-known object entry with the GUID "6c01d2a7-f083-4503-8132-789eeb127b84", which is on the "CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=corp,DC=Company,DC=com" container object's otherWellKnownObjects attribute, refers to a group "CN=Exchange Servers,OU=Microsoft Exchange Security Groups,DC=corp,DC=Company,DC=com" of the wrong group type. Either delete the well-known object entry, or promote the target object to "Universal, SecurityEnabled".
[04/27/2012 02:38:43.0487] [1] 0.  ErrorRecord: Microsoft.Exchange.Management.Tasks.InvalidWKObjectTargetException: The well-known object entry with the GUID "6c01d2a7-f083-4503-8132-789eeb127b84", which is on the "CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=corp,DC=Company,DC=com" container object's otherWellKnownObjects attribute, refers to a group "CN=Exchange Servers,OU=Microsoft Exchange Security Groups,DC=corp,DC=Company,DC=com" of the wrong group type. Either delete the well-known object entry, or promote the target object to "Universal, SecurityEnabled".
[04/27/2012 02:38:43.0487] [1] [ERROR] The following error was generated when "$error.Clear();
initialize-ExchangeUniversalGroups -DomainController $RoleDomainController -ActiveDirectorySplitPermissions $RoleActiveDirectorySplitPermissions
" was run: "The well-known object entry with the GUID "6c01d2a7-f083-4503-8132-789eeb127b84", which is on the "CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=corp,DC=Company,DC=com" container object's otherWellKnownObjects attribute, refers to a group "CN=Exchange Servers,OU=Microsoft Exchange Security Groups,DC=corp,DC=Company,DC=com" of the wrong group type. Either delete the well-known object entry, or promote the target object to "Universal, SecurityEnabled".".
[04/27/2012 02:38:43.0487] [1] [ERROR] The well-known object entry with the GUID "6c01d2a7-f083-4503-8132-789eeb127b84", which is on the "CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=corp,DC=Company,DC=com" container object's otherWellKnownObjects attribute, refers to a group "CN=Exchange Servers,OU=Microsoft Exchange Security Groups,DC=corp,DC=Company,DC=com" of the wrong group type. Either delete the well-known object entry, or promote the target object to "Universal, SecurityEnabled".
[04/27/2012 02:38:43.0487] [1] [ERROR-REFERENCE] Id=443949901 Component=
[04/27/2012 02:38:43.0487] [1] Setup is stopping now because of one or more critical errors.
[04/27/2012 02:38:43.0487] [1] Finished executing component tasks.
[04/27/2012 02:38:43.0502] [1] Ending processing Install-ExchangeOrganization
[04/27/2012 02:38:43.0502] [0] The Exchange Server setup operation didn't complete.  More details can be found in ExchangeSetup.log located in the <SystemDrive>:\ExchangeSetupLogs folder.
[04/27/2012 02:38:43.0502] [0] End of Setup
Troubleshooting:
-    The problem is happening because of some issue with the USGs in the Microsoft Exchange Security Groups container.
-    Opened LDP.exe
-    Accessed the Configuration partition.
-    Right clicked on Microsoft Exchange.
-    Clicked modify option.
-    Pasted the  B:32:A7D2016C83F003458132789EEB127B84:CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=corp,DC=Company,DC=com in the value.
-    Clicked on Delete Radio button.
-    Clicked on Run.
-    Forced AD replication.
-    Ran Setup /prepareAD again.
-    It again failed with the same message but this time the Group is different.
-    Noticed that a new ExchangeServers1 group has been created.
-    We wanted to check why this issue happening for multiple groups.
-    So we compared the LDP dump of the old Exchange Servers and the new Exchange Servers1 group.
-    The old group was mail enabled. It had exchange attributes like legacyExchangeDN, proxyAddresses, mail, mailnickname etc...
-    Checked the GAL and Exchange Servers group is visible there.
-    We have no idea how this happened or who mail enabled these groups.
-    Same is the case with other Groups in that OU as well.
-    We removed the corresponding entries for the remaining groups from CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=corp,DC=Company,DC=com.
-    Forced AD Replication and ran setup /prepareAD again.
-    This time setup succeeded.
-    We successfully installed the CAS role as well.
-    Powershell is not connecting to the local server.
-    We are getting the below error.
========
The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol.
========
-    Checked the bindings on Default Web Site from IIS manager.
-    HTTP was bound to a specific IP rather than All Unassigned.
-    Changed it to All Unassigned and restarted IIS.
-    Powershell started working fine.
-    We are having issues with Management roles for admins.
-    The Server Configuration section is not visible in EMC and we are not able to see user properties from EMC.
-    Get-Mailbox returns only the logged on user's details even though the logged on user is an ORG Admin.
-    This looks like a RBAC issue.
-    Checked AD permissions
-    The admin user has Full Control on all containers.
-    Still he is not able to Administer user accounts from EMS or Shell.
-    Tested ECP access and it also failed with Access Denied.
-    Identified http://social.technet.microsoft.com/Forums/en-US/exchange2010/thread/8f9a1881-d66d-4d8a-a6ff-06729a701999
-    We followed the below steps to resolve the issue.
==============
Do the following on the Exchange 2010 server using the same account used to install Exchange 2010.
1)     Open Windows PowerShell (not the Exchange Management Shell)
a.       If you have UAC enabled, right click Windows PowerShell and click Run as administrator.
2)     Run Start-Transcript c:\RBAC.txt and press enter
a.       This will start logging all commands and output you type to a text file.
3)     Run Add-PSSnapin *setup and press enter
a.       This adds the setup snap-in which contains the setup cmdlets used by Exchange during install. You may see errors about loading a format data file. You can ignore those errors.
DO NOT run any other cmdlets in this snap-in without direction from Microsoft. Doing so could irreparably damage your Exchange installation.
4)     Run Install-CannedRbacRoleAssignments -InvocationMode Install -Verbose and press enter.
a.       This cmdlet should create the required role assignments between the role groups and roles that should have been created during setup.
b.       Be sure you run with the Verbose switch so we can capture what the cmdlet does.
5)     Run Remove-PSSnapin *setup and press enter
6)     Run $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri http://<FQDN of Exchange 2010 server>/PowerShell/ -Authentication Kerberos and press enter
a.       Be sure to replace <FQDN of Exchange 2010 server> with the FQDN of your server.
7)     Run Import-PSSession $Session and press enter
8)     Run Get-ManagementRoleAssignment and press enter
9)     Run Stop-Transcript and press enter
=============
-    This resolved the RBAC issue as well.





2012/05/03

Lync TrustModelData

The Issue

Recently, I had a small issue with Lync 2010 clients objecting to the certificate on the autodiscover.company.com CAS server.

The internal domain in use for SIP was company.local, and the CAS had company.com, although autodiscover.company.local was also on that cert.

Interestingly, we first looked at strict DNS naming, however, that did not appear to be the root cause. A colleague pointed out to me that he thought the TrustModelData might be doing it, so we investigated that route.

The error popped up as soon as autodiscover.company.com (the SMTP domain) was added to DNS. Here is the error :

image

Lync is hardcoded to act on finding Autodiscover, and it immediately attempts to connect to the advertised EWS.  In our case, the SMTP domain does not match the SIP domain (company.com v company.local).

The Fix

As it turns out, there is a registry entry in HKLM to control this behavior – by default this key is populated with a selection of Microsoft Online entries – none of which matched our company.local. We pushed the following registry change with SCCM; GPO was not an option due to XP workstations. 

This option is NOT part of Lync in-band client provisioning, and you can put the entry in either

HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Communicator\TrustModelData 

OR

HKEY_CURRENT_USER\Software\Policies\Microsoft\Communicator TrustModelData

and APPEND your SMTP domain to the list.

SNAGHTMLf6ce697

The precedence for applying these changes is:

(1) In-band provisioning, (2) HKLM, (3) HKCU, (4) Lync option set in client. 

Because of this ordering, we decided on using HKLM for our registry change so that all users of the workstation would get the change.

YMMV.

test 02 Feb

this is a test it’s only a test this should be a picture