Creating AlwaysOn Availability Group - SQL Server 2016 -Step By Step

AlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize availability for one or more user databases. AlwaysOn Availability Groups requires that the SQL Server instances reside on Windows Server Failover Clustering (WSFC) nodes.

Prerequisites

• Ensure that the system is not a domain controller.
• Ensure that each computer is running Windows Server 2012 or later versions.
• Ensure that each computer is a node in a Windows Server Failover Clustering (WSFC) cluster.

My Environment 

OS - Windows 2012 R2

SQL Server - SQL Server 2016 Enterprise Edition (Eval)

Fail-over Cluster Installation

First we need to add the Windows Fail-over Cluster Feature to all the nodes running the SQL Server instances that we will configure as replicas.

Open the Server Manager  => select Add roles and features 

Click Next till Select Features dialog box.

    Select the Failover Clustering checkbox

Click Install to install the Failover Clustering feature.

Windows Failover Clustering Configuration for AlwaysOn Availability Groups

Open Server Manager and select "Fail over Cluster Manager"

Select Fail over Cluster Manager and Select the Option Create Cluster OR 

You can Right Click on the Fail over Cluster manager and select Create Cluster menu

You will Create Cluster Wizard

 Add the Nodes you want be in the Cluster and then click next

You will be getting a validation warning , Select Yes to Validate the Cluster nodes

 Select the option -  Run all tests

NOTE: The Cluster Validation Wizard is expected to return several Warning messages, especially if you will not be using shared storage. other than that if you find any Error messages, you need to fix  prior to creating the Windows Server Failover Cluster.

In the Access Point for Administering the Cluster dialog box, enter the Cluster name and virtual IP address for Windows Server Failover Cluster , then Click next.

Verify that the configuration is successful in Summary.

  Configure Cluster Quorum Settings

Quorum is that it is a configuration database for the cluster and is stored on a shared location, accessible to all of the nodes in a cluster.

In Case of Even number of nodes (but not a multi-site cluster) Node and Disk Majority Qurum configuration is recommended . If you dont have no shared storage Node and File Share Majority is recommended. Here it will be configuring a FileShare Witness quorum


It is recommended that you configure the quorum size to be 500 MB; this size is the minimum required for an efficient NTFS partition. Larger sizes are allowable but are not currently needed.

Click on Your Cluster Select Configure Cluster Quorum Setting from the More action menu

 In the Select Quorum Witness page, select the Configure a file share witness option. Click Next.

Type path of the file share that you want to use in the File Share Path: text box. Click Next.
It is recommended to have the share folder on a different node than the node participating on Cluster

 In the Confirmation page, click Next.

In the Summary page, click Finish.

Enable AlwaysOn Availability Groups Feature on  SQL Server 2016

we can now proceed with enabling the AlwaysOn Availability Groups feature in SQL Server 2016. This is possible after install and configuring the Windows Fail-over Cluster on all the nodes

Open SQL Server Configuration Manager - > SQL Server Properties - [SQL Server (SQLAG01)]

In the Properties dialog box, select the AlwaysOn High Availability tab. Check the Enable AlwaysOn Availability Groups check box. This will prompt you to restart the SQL Server service. Click OK.
 
Restart the SQL Server service.

Configure SQL Server 2016 AlwaysOn Availability Groups

 Go to Management Studio, right click Availability Groups and click New Availability Group Wizard.

Specify Availability Group Name .  my group name SQLAVG2016 then click Next.

Choose Database

Here you can see whether the db meets the prerequisites

• Database should be in full recovery mode. 
• You should make a full backup to add the DB into the Availability Group

Specify Replicas .
This page applies to the New Availability Group Wizard and the Add Replica to Availability Group Wizard of SQL Server 2016

If a server instance that you to use to host a secondary replica is not listed by the Availability Replicas grid, click the Add Replica button.

Add Azure Replica button to create virtual machines with secondary replicas in Windows Azure.

Adding Secondary Replica

Endpoints

Use this tab to verify any existing database mirroring endpoints and also, if this endpoint is lacking on a server instance whose service accounts use Windows Authentication, to create the endpoint automatically.

Backup Preference

Use this tab to specify your backup preference for the availability group as a whole and your backup priorities for the individual availability replicas.

Listener

An availability group listener is a virtual network name (VNN) to which clients can connect in order to access a database in a primary or secondary replica of an AlwaysOn availability group. You point applications to the listener (which is registered with DNS) and directs traffic in the AG

Select Data Synchronization

Use the Always On Select Initial Data Synchronization page to indicate your preference for initial data synchronization of new secondary databases. This page is shared by three wizards—the New Availability Group Wizard, the Add Replica to Availability Group Wizard, and the Add Database to Availability Group Wizard.

The possible choices include Full, Join only, or Skip initial data synchronization. Before you select Full or Join only ensure that your environment meets the prerequisites.

In My I have choose Full 

For each primary database, the Full option performs several operations in one workflow: create a full and log backup of the primary database, create the corresponding secondary databases by restoring these backups on every server instance that is hosting a secondary replica, and join each secondary database to availability group.

Select this option only if your environment meets the following prerequisites for using full initial data synchronization, and you want the wizard to automatically start data synchronization

 

 

AlWaysOn Availability Group - Create New Availability Group Error - Checking for compatibility of the database file location on the Server Instance that hosts secondary replica (Microsoft.SqlServer.Management.HadrModel)

This is one of common error message While Creating a New Availability Group   Here is full error I received when I attempted to Create a New Availability Group.

Here is my environment

OS - Windows Server 2012 R2
SQL Server - SQL 2016 Enterprise Edition ( The same solution will solve for SQL 2014/2012 too)
Primary Server - SQLAG01
Secondary Server - SQLAG02


We used ‘Full’ option in Select Initial Data Synchronization Page.  The error says Primary replica is trying to search for data folder location in secondary and as a result they don’t match, and failed.

If you have choosen Full option  You have to create same file location on both the instances
e,g: Create Same Data path on both instances.

Data Path on SQLAG01 -  "C:\Program Files\Microsoft SQL  Server\MSSQL13.SQLAG01\MSSQL\DATA\"

Data Path on SQLAG02- "C:\Program Files\Microsoft SQL  Server\MSSQL13.SQLAG01\MSSQL\DATA\"

other wise choose other options for Initial Data synchronization ' Join Only' and manually restore the database on secondary

Add Databases to AlwaysOn Availability Group Error - The operating system returned the error '5(Access is denied.)

This is one of common error message While Creating a New Availability Group or Adding a Database to an availability group. Here is full error I received when I attempted to Add a database to an Availability Group

--------------------------------------------------------------------------------------------------------------------------------------------------------

Restore Database AdventureWorks fail (Microsoft.SqlServer.Management.HadrModel)

ADDITIONAL INFORMATION:

Restore failed for Server 'CGSQL02\SQLAG02'.  (Microsoft.SqlServer.SmoExtended)

System.Data.SqlClient.SqlError: The operating system returned the error '5(Access is denied.)' 

while attempting 'RestoreContainer::ValidateTargetForCreation' on 'C:\Program Files\Microsoft SQL Server\MSSQL13.SQLAG01\MSSQL\DATA\AdventureWorks.mdf'. (Microsoft.SqlServer.Smo)

--------------------------------------------------------------------------------------------------------------------------------------------------------

Solution

Here is my environment

OS - Windows Server 2012 R2
SQL Server - SQL 2016 Enterprise Edition ( The same solution will solve for SQL 2014/2012 too)
Primary Server - SQLAG01 

Secondary Server - SQLAG02

SQL Installation was done on Default Paths  

Data Path on SQLAG01 -  "C:\Program Files\Microsoft SQL  Server\MSSQL13.SQLAG01\MSSQL\DATA\"

Data Path on SQLAG02- "C:\Program Files\Microsoft SQL  Server\MSSQL13.SQLAG02\MSSQL\DATA\"

SQL Service is running under  domain acccount ( condoso/sql_service )

it is a problem with the Permission of the specified folder to the user account that SQL Server service is running. The Path Which you have created on Secondary instance to perform synchronization of secondary server databases using backup/restore. in my scenario ( "C:\Program Files\Microsoft SQL  Server\MSSQL13.SQLAG01\"  Which is the Path Created for Back/restore on Secondary node ) , This path should have Full Permission on SQL Service Account. condoso\sql_service

AlWays on Availability group and Fileshare witness Quorum , Windows 2012 & SQL Server 2012

AlWays on Availability group and Fileshare witness Quorum

What is AlWaysOn Availability Group

AlwaysOn Availability Groups is an enterprise-level high-availability and disaster recovery solution introduced in SQL Server 2012 to enable you to maximize availability for one or more user databases. AlwaysOn Availability Groups requires that the SQL Server instances reside on Windows Server Failover Clustering (WSFC) nodes.

What is a Quorum in Failover Cluster

Quorum is that it is a configuration database for the cluster and is stored on a shared location, accessible to all of the nodes in a cluster.

Different Type of Quorum

Node Majority (recommended for clusters with an odd number of nodes)

Can sustain failures of half the nodes (rounding up) minus one. For example, a seven node cluster can sustain three node failures.

Node and Disk Majority (recommended for clusters with an even number of nodes)

Can sustain failures of half the nodes (rounding up) if the disk witness remains online. For example, a six node cluster in which the disk witness is online could sustain three node failures.

Can sustain failures of half the nodes (rounding up) minus one if the disk witness goes offline or fails. For example, a six node cluster with a failed disk witness could sustain two (3-1=2) node failures.


Node and File Share Majority (for clusters with special configurations)

Works in a similar way to Node and Disk Majority, but instead of a disk witness, this cluster uses a file share witness.

Note that if you use Node and File Share Majority, at least one of the available cluster nodes must contain a current copy of the cluster configuration before you can start the cluster. Otherwise, you must force the starting of the cluster through a particular node. For more information, see “Additional considerations” in Start or Stop the Cluster Service on a Cluster Node.

No Majority: Disk Only (not recommended)

Can sustain failures of all nodes except one (if the disk is online). However, this configuration is not recommended because the disk might be a single point of failure.


Windows 2012:


It is continued to improve quorum On Windows 2012.  In a majority node set, as soon as you lose a majority of your votes, the remaining nodes go offline. So for example, if you have a cluster with FIVE nodes, if you were to lose THREE nodes the remaining nodes will go offline, even though there are three nodes remaining. This might not be exactly what you want to happen. So in Windows Server 2012, Microsoft introduced Dynamic Quorum.
The quorum models are the same in Windows 2012 like Windows 2008 R2, however there are enhancements with Node Vote Assignment and there is a new concept called as “Dynamic Quorum Configuration”.


Dynamic Quorum Configuration:

Starting Windows Server 2012, there is a new concept called dynamic quorum configuration. This concept is based on node vote assignment, specifically it relies on the fact that cluster can manage node vote assignment automatically based on the state of each node.  Votes are automatically removed from nodes that leave active cluster membership, and a vote is automatically assigned when a node rejoins the cluster. By default, dynamic quorum management is enabled.

In above scenario of five node cluster When node one went offline, we have four node cluster. When node two went offline, I would then have a THREE node cluster, and so on. In reality, It continues to lose cluster nodes one by one, all the way down to a two node cluster and still remain online. And, if I had configured a witness (Disk or File Share) I could actually go all the way down to a single node and still remain online.

However, best practice in Windows Server 2012 and earlier is to only configure a witness on clusters that have an even number of nodes. Here is an example of why you wouldn’t configure a witness in cluster with an odd number of nodes. Fore example you have a three node cluster and decide to configure a witness (Disk or File Share). If you happen to lose the Witness and a single node, the remaining two nodes will go offline because they only have two out of four votes, not a majority. So adding a witness to a cluster that already has an odd number of votes actually gives you less availability than if you didn’t add a witness at all.

That leads Windows Server 2012 R2 and Dynamic Witness. In the scenario above on Windows Server 2012,  However, on Windows Server 2012 R2 has changed to ALWAYS configure a witness, regardless of the number of nodes. The cluster itself will determine whether the witness actually has a vote, depending upon how many cluster nodes are available at any given time. So in the example above with five node cluster, still configure a witness. Under normal circumstances when all FIVE nodes are running, witness would not have a vote. However, as a started losing cluster nodes, the witness vote would be turned on or off as needed to ensure that I always had an odd number of votes in my cluster.

How can I decide Which quorum is  good and What is the recommended size of the File Share/Disk Witness Quorum ?

In Case of Even number of nodes (but not a multi-site cluster) Node and Disk Majority Qurum configuration is recommended . If you dont have no shared storage Node and File Share Majority is recommended. Requires One quorum disk (or File Share)  per cluster.

It is recommended that you configure the quorum disk size to be 500 MB; this size is the minimum required for an efficient NTFS partition. Larger disk sizes are allowable but are not currently needed.


Reference
https://blogs.msdn.microsoft.com/microsoft_press/2014/04/28/from-the-mvps-understanding-the-windows-server-failover-cluster-quorum-in-windows-server-2012-r2/

SQL Server Transact SQL - TSQL - TOP WITH TIES

TOP Keyword is commonly used to get the Top Rows from a Result. SQL Server has option use it 'WITH TIES' to get similar rows together

WITH TIES
Used when you want to return two or more rows that tie for last place in the limited results set. Must be used with the ORDER BY clause. WITH TIES may cause more rows to be returned than the value specified in expression. For example, if expression is set to 5 but 2 additional rows match the values of the ORDER BY columns in row 5, the result set will contain 7 rows.

SELECT TOP (5) productid, unitprice FROM Production.Products
WHERE categoryid = 1 ORDER BY unitprice DESC;

You get the following result set.


productid                unitprice
----------- ---------------------
38                           263.50
43                             46.00
2                               19.00
1                               18.00
35                             18.00

If you want provide solutions to turn this query into a deterministic one—one solution that includes ties and another that breaks the ties. First, address the version that includes all ties by using the WITH TIES option. Add this option to the query, as follows.

SELECT TOP (5) WITH TIES productid, unitprice FROM Production.Products
WHERE categoryid = 1 ORDER BY unitprice DESC;

You get the following output, which includes ties.
productid                unitprice
----------- ---------------------
38                           263.50
43                             46.00
2                               19.00
1                               18.00
35                             18.00
76                             18.00

AlWaysOn Availability Group Enhancements in SQL Server 2016

• AlWaysOn Feature is there available since SQL Server 2012 and There are some enhancements in SQL Server 2014 is the increased maximum number of secondaries. SQL Server 2012 supported a maximum of four secondary replicas. AlwaysOn Availability Groups on 2014 supports up to eight secondary replicas. Also use SQL Server 2014 AlwaysOn Availability Groups to provide high availability for SQL Server databases hosted in Windows Azure etc .

• Database Level Fail over Trigger SQL Server 2016 is coming with solving pains on Failover , Currently Failover is happening based on instance health, on 2016 it can fail-over based on Database health as well. If any DB in an availability group fails the whole Group of Database will fail over to other. 

• More Auto fail-over Targets : Another Enhancement is increase in number of Automatic Failover partners , Which means you can have two more fail-over partners apart from Primary. In this 3 node Scenario , one node fails still we can have high availability with other two nodes.

• Domain independent Availability Group You can create an availability group in Work Group Computers .Earlier All the nodes which is participating in a availability group must be in same domain

• Load balancing on Availability group secondaries , Earlier Only first replica in secondaries get all the read only traffic even if you have more than one replicas. On 2016 you can Distribute the Workload for read only transaction across multiple secondaries 

• Distributed transactions are supported with AlwaysOn Availability Groups. 

This applies to distributed transactions between databases hosted by two different SQL Server instances. It also applies to distributed transactions between SQL Server and another DTC-compliant server.

The following requirements must be met:

Availability Groups must be running on Windows Server 2016 or Windows Server 2012 R2. For Windows Server 2012 R2, you must install the update in KB3090973 available at https://support.microsoft.com/en-us/kb/3090973.

Availability Groups must be created with the CREATE AVAILABILITY GROUP command and the WITH DTC_SUPPORT = PER_DB clause. You cannot currently alter an existing Availability Group.

• Basic Availability Group Feature will be supported by Standard Edition of SQL Server ( 2 Replicas Only and One Database per Group) , Basic Feature wont have readable secondary , no Backup on secondary replica etc , for those you have to choose advanced Availability group feature on Enterprise Edition. Whole idea behind basic Availability group is to retire Mirroring in Future
• Mirroring wont be deprecating on 2016 May be in future versions

The database transaction log file increasing for the DBs Participating In AlwaysOn Availability Groups

Issue

After Migrating High Available SQL Server 2012 Cluster Instance to SQL Server 2014  AlWays on Availability Group  transaction Log file keep growing and within couple of weeks almost the hard disk is full. Earlier My Db was in Simple recovery mode , I changed it to Full recovery mode to add it on Availability Group because of that the transaction log keeps increasing

Solution

It is  a very common problem the setting database recovery mode to FULL and then forgetting to backup the transaction log (LDF file). Let me explain how to fix it.

If you ready loose  a some data between backups, just set the database recovery mode to SIMPLE, then forget about LDF - it will be small. the solution for most of the cases. But for SIMPLE Recovery mode will not supporting by Availability group. In this case you have to take Transaction Log backups.
Recommended to have  a full backup per day. If you do a full backup per day, The frequency of the log backup will determine how much data you are allowed to lose in case of a failure. If you run your log backup every 15 minutes, expect to loose up to the last 15 minutes of data that changed. 15 minutes was good frequency for my environment.


AlwaysOn Availability Groups allows the offloading backups to a secondary replica. If you setup a log backup on secondary replica it will be truncating both primary and secondary logs. no need to have it on both nodes.

Where should backups occur? select the automated backup preference for the availability group, one of:

Prefer Secondary 

Specifies that backups should occur on a secondary replica except when the primary replica is the only replica online. In that case, the backup should occur on the primary replica. This is the default option.

Secondary only

Specifies that backups should never be performed on the primary replica. If the primary replica is the only replica online, the backup should not occur.

Primary

Specifies that the backups should always occur on the primary replica. This option is useful if you need backup features, such as creating differential backups, that are not supported when backup is run on a secondary replica.

Any Replica

Specifies that you prefer that backup jobs ignore the role of the availability replicas when choosing the replica to perform backups. Note backup jobs might evaluate other factors such as backup priority of each availability replica in combination with its operational state and connected state.

To take the automated backup preference into account for a given availability group, on each server instance that hosts an availability replica whose backup priority is greater than zero (>0), you need to script backup jobs for the databases in the availability group. To determine whether the current replica is the preferred backup replica, use the sys.fn_hadr_backup_is_preferred_replica function in your backup script. If the availability replica that is hosted by the current server instance is the preferred replica for backups, this function returns 1. If not, the function returns 0. By running a simple script on each availability replica that queries this function, you can determine which replica should run a given backup job.

If you use the Maintenance Plan Wizard to create a given backup job, the job will automatically include the scripting logic that calls and checks the sys.fn_hadr_backup_is_preferred_replica function. However, the backup job will not return the “This is not the preferred replica…” message. Be sure to create the job(s) for each availability database on every server instance that hosts an availability replica for the availability group.
:

Common table expressions CTE - SQL Server

Common table expression is a similar concept of a derived table in the sense that it’s a named table expression that is visible only to the statement .

A derived table is basically a subquery, except it is always in the FROM clause of a SQL statement. The reason it is called a derived table is because it essentially functions as a table as far as the entire query is concerned.

A common table expression (CTE) can be thought of as a temporary result set that is defined within the execution scope of a single SELECT, INSERT, UPDATE, DELETE, or CREATE VIEW statement. A CTE is similar to a derived table in that it is not stored as an object and lasts only for the duration of the query. Unlike a derived table, a CTE can be self-referencing and can be referenced multiple times in the same query.
A CTE can be used to:
Create a recursive query. For more information, see Recursive Queries Using Common Table Expressions.
Substitute for a view when the general use of a view is not required; that is, you do not have to store the definition in metadata.
Enable grouping by a column that is derived from a scalar subselect, or a function that is either not deterministic or has external access.
Reference the resulting table multiple times in the same statement.




Common Table Expression (CTE)  contains  three parts:

a) The name Starts  WITH keyword
b) List of Columns (optional)
c) Query (in parentheses after the AS keyword)

The query using the CTE must be the first query appearing after the CTE.

You have a query like this

SELECT * FROM (
SELECT A.productid, A.productname, E.categoryname,a.unitprice From Production.Products A Inner join Production.Categories E on E.categoryid = A.categoryid) T
WHERE T.unitprice > 50
ORDER BY T.categoryname

rewriting the query using CTE expression as follows

With T(productid, productname,categoryname, unitprice)
AS
(
SELECT P.productid, P.productname, C.categoryname,P.unitprice From Production.Products P
Inner join Production.Categories C on C.categoryid = P.categoryid
)
SELECT * FROM T
WHERE T.unitprice > 50
ORDER BY T.categoryname

PIVOT and UNPIVOT - SQL Server

Pivoting is a grouping and aggregating of data. Unpivoting is, in a sense, the inverse of pivoting.PIVOT rotates data from a state of rows to a state of columns; UNPIVOT rotates the data from columns to rows.

WITH PivotData AS
(
SELECT
< grouping column >,
< spreading column >,
< aggregation column >
FROM < source table >
)
SELECT < select list >
FROM PivotData
PIVOT( < aggregate function >(< aggregation column >)
FOR < spreading column > IN (< distinct spreading values >) ) AS P;

To understand PIVOT, let us create a Temporary Table #Books with sample records in below script:

if object_id('tempdb..#Books') is not null
drop table #Books
GO

CREATE TABLE #Books
(Book VARCHAR(150),Year INT,Price MONEY)
GO
INSERT INTO #Books VALUES('SQL 2012',2012,1300)
INSERT INTO #Books VALUES('Oracle',2012,1200)
INSERT INTO #Books VALUES('Azure',2012,1500)
INSERT INTO #Books VALUES('Azure',2014,1800)
INSERT INTO #Books VALUES('Oracle',2014,1300)

SELECT * FROM #Books PIVOT(SUM(Price ) FOR Book IN ([Azure], Oracle)) AS PVTTable

SELECT * INTO #BookPriceinResult FROM #Books PIVOT(SUM(price)
FOR book IN ([Azure], [Oracle])) AS PVTTable;

select * from #BookPriceinResult;

SELECT Book, Year, Price FROM #BookPriceinResult
UNPIVOT(Price FOR Book IN ([Azure], Oracle)) AS UNPVTTable;

The permissions granted to user 'domain\username' are insufficient for performing this operation. (rsAccessDenied) SQL Server Report Server (SSRS)

Reporting Services uses role-based security to grant user access to a report server. On a new report server installation, only users who are members of the local Administrators group have permissions to report server content and operations. To make the report server available to other users, you must create role assignments that map  user or group accounts to a predefined role that specifies a collection of tasks.

Verify that sufficient permissions have been granted and windows user account control (uac) restrictions have been addressed

It is an insufficient  privilege for current user you are running the report, give that user a privilege to run report / report folders

Start Internet Explorer As "Run as Administrator"
Open Your Report Server URL (http://'ServerName'/'ReportFolder')
Go to Site Settings

Security->New Role Assignment

Add DOMAIN/USERNAME or DOMAIN/USERGROUP

Step 2

Go to Folder Settings

Security->New Role Assignment

Add DOMAIN/USERNAME or DOMAIN/USERGROUP
Check the Report URL