Browsing: SQL Server DBA Tutorial: Beginners to Pro

Understanding SQL Server Database Files and Filegroups

Hi there! Here we are going to build a complete Understanding of SQL Server Database Files and Filegroups. This is another basic and one of the important topics to help you in your SQL DBA carrier. This is part of SQL Server DBA Tutorial: Beginners to Pro series.

What are SQL Server Database Files:

Every SQL Server database is made up of at least two operating system files: a data file and a log file.

  • The data file stores data and database objects such as tables, indexes, stored procedures, and views.
  • The log file records the information required to recover all transactions in the database.
Demo: Create database with Default settings and check the files

Create a database with default settings with the following basic T-SQL script.

 CREATE Database Testdb

Now in SSMS Right Click on the Database > Properties > Click on Files

SQL Server Database Files and Filegroups

Types of SQL Server Database Files:

SQL Server Database have three main types of database files:

  1. Primary Data File (.mdf)
  2. Secondary Data File (.ndf)
  3. Transaction Log File (.ldf)
Primary Data File (.mdf):
  • Contains startup information for the database.
  • Stores system tables and metadata.
  • It points to the other files in the database.
  • Every database has only one primary data file.
  • Microsoft recommends the file name extension for primary data files is .mdf.
Secondary Data File (.ndf)
  • This is optional user-defined data file.
  • SQL Database can have multiple secondary files.
  • Data can spread across multiple disks by putting each file on a different disk drive. Improve performance via parallel I/O.
  • Microsoft recommends file name extension for secondary data files is .ndf.
Transaction Log File (.ldf):
  • Stores all transactions and changes.
  • Essential for recovery of the database.
  • Each database has at least one log file.
  • Recommended file name extension for transaction logs is .ldf.

Logical and Physical File Name:

There are two types of File Names if you have observed in the above screen shot. The first column says Logical Name and the last column labeled as File Name with extensions .mdf and .ldf are Physical File Name.

Logical File Name: 
  • The file name which is defined when the database is created.
  • The name used to refer to the physical file in all Transact-SQL statements (BACKUP, RESTORE, CREATE DATABASE, ALTER DATABASE, etc.)
  • The logical file complies with the rules of SQL Server identifiers.
  • Logical file name must be unique among all logical file names in the database.

e.g. In the CREATE Database T-SQL, Testdb is the logical file Name

 CREATE Database Testdb  --Testdb is the Logical File Name of the Database
Physical File Name:
  • Physical file name is the actual name of the file stored on disk.
  • Represents the real .mdf, .ndf, or .ldf file.
  • The physical file name complies with the rules of Operation System identifiers.

Example: Let’s display the logical and physical file names for the database ”Testdb” created above

 USE Testdb 
 GO
 SELECT name as logical_name, physical_name FROM sys.database_files
Result Set showing logical and physical file names:

SQL Server Files and Filegroups

What Is Filegroup:

A Filegroup is a logical container of the database files. It allows you to control how and where database objects (like tables and indexes) are physically stored on disk.

  • Helps in data allocation and administration
  • Enables performance optimization by spreading data across disks

Types of Filegroups:

  • Primary Filegroup
  • User-defined
  • Filestream
  • Memory Optimized Data
Primary Filegroup:
  • Primary filegroup contains the primary data file (.mdf).
  • It holds the system objects and tables.
  • It is the default filegroup.

Bear in mind that you can change the default filegroup using T-SQL so that new objects are created in the user defined filegroup. However system objects and tables remains with the Primary Filegroup.

Example: Let’s check the filegroup information for the database “Testdb” created above.

Right Click on the Database > Properties > Click on Files

SQL Server Files and Filegroups

As can be seen above, the default Filegroup for TestDB data file is showing as PRIMARY. Point often overlooked is the Filegroup for TestDB_log file is Not Applicable.

Hence, keep in mind that SQL Server does not allow you to place transaction log files in any filegroup

User-defined Filegroup:
  • As the name suggests, DBAs create user-defined filegroups.
  • Used to separate large tables, indexes, or archive data.
Filestream Filegroup:
  • Stores unstructured data (e.g., documents, images).
Memory Optimized Data:
  • Used for In-Memory OLTP tables.
  • To create memory-optimized tables, you must first create a memory-optimized filegroup.
Demo: The following T-SQL script shows how to create primary File on PRIMARY Filegroups and Secondary Files on User-Defined Filegroup
T-SQL Script

USE master;
GO

CREATE DATABASE FileGroupDemo
ON PRIMARY -- Primary Filegroup
(
    NAME = N'FileGroupDemo_Primary', -- Logical Filename of Primary File
    FILENAME = N'C:\SQLData\FileGroupDemo_Primary.mdf',  -- Primary Physical Filename 
    SIZE = 100MB,
    MAXSIZE = 200MB,
    FILEGROWTH = 10MB
),
FILEGROUP FG_Sales  --User-Defined Filegroup with name FG_Sales 
(
    NAME = N'FileGroupDemo_Sales_01',  --Logical Filename of Secondary File 
    FILENAME = N'C:\SQLData\FileGroupDemo_Sales_01.ndf',  --Physical filename of Secondary File 
    SIZE = 20MB,
    MAXSIZE = 200MB,
    FILEGROWTH = 10MB
),
FILEGROUP FG_Archive  --User-Defined Filegroup with name FG_Archive 
(
    NAME = N'FileGroupDemo_Archive_01', --Logical Filename of Secondary File 
    FILENAME = N'C:\SQLData\FileGroupDemo_Archive_01.ndf',  --Physical filename of Secondary File 
    SIZE = 20MB,
    MAXSIZE = 200MB,
    FILEGROWTH = 10MB
)
LOG ON
(
    NAME = N'FileGroupDemo_Log', --Logical filename of Transaction Log File 
    FILENAME = N'C:\SQLData\FileGroupDemo_Log.ldf', --Physical filename of Transaction Log File 
    SIZE = 10MB,
    MAXSIZE = 100MB,
    FILEGROWTH = 5MB
);
  

Now let’s see the database properties:

SQL Server Files and Filegroups

Hope you got a fair idea about SQL Server File and Filegroups. We’ll discuss about Filestream and Memory Optimized Filegroup in next article. Please feel free to leave a comment.

You may also like to go through:

What is SQL Server

Understanding SQL Server Versions and Editions

What Does SQL DBA Actually Do?

The Complete Guide to SQL Server System Databases

{ Add a Comment }

What Is SQL Server Transaction: Complete Beginner’s Guide

Hi There! This article is to explain What Is SQL Server Transaction and this is part of the SQL Server DBA Tutorial: Beginners To Pro category. This will help in your journey to become a better DBA.

What is SQL Server Transaction?

A SQL Server transaction is a sequence of one or more SQL statements that are executed together. The transaction ensures that either all operations succeed or none of them are applied to the database.

In other words, batch of SQL Statement(s) performing a single or multiple activities on a database. Transactions ensures data integrity by either of the below:

  • Committing all changes together
  • Rolls back changes if any operation fails.

Example 1:

What is SQL Transaction

Example 2:

What is SQL Transaction

 

Practical Example: Bank Money Transfer:

Suppose you are transferring money from Account A to Account B.
Both operations must succeed together or fail. I.e. Either money should get credited to Account B or the money should not debit from Account A.

Different Modes Of SQL Server Transactions:

  • Auto-commit transactions
  • Implicit transactions
  • Explicit transactions
  • Batch-scoped transactions

Auto-commit transactions:

Each individual statement is a transaction. The auto-commit transaction mode is the default SQL Server transaction mode. When an SQL statement executes successfully, the system commits the transaction and permanently saves the data modifications in the database. If any statement encounters an error during execution, the system rolls back the transaction and reverts all changes made during that transaction.

Example:

 INSERT INTO Products (ProductID, ProductName, Price)
 VALUES (1,'Laptop',1200)

 UPDATE Products SET Price = 1300 WHERE ProductID = 1

 DELETE FROM Products WHERE ProductID = 1

As you must have observed that there is no BEGIN TRAN statement mentioned in the above example and hence each statement (INSERT, UPDATE, DELETE) runs as its own transaction.

Imagine a scenario when INSERT and UPDATE completes successfully but the DELETE statement fails, only the DELETE is rollback, the changes for INSERT and UPDATE remains saved.

Implicit transactions

In SQL Server, an Implicit Transaction starts automatically when you execute certain SQL statements. The transaction does not commit automatically; you must explicitly end it using COMMIT or ROLLBACK.

Let’s understand Implicit transaction by coding example:

Create a Table: 
CREATE TABLE Products ( 
    ProductID INT PRIMARY KEY, 
    ProductName VARCHAR(50),
    Price DECIMAL(10,2)
)
Enable Implicit Transaction Mode:
SET IMPLICIT_TRANSACTIONS ON;
Commit an Implicit Transaction:
INSERT INTO Products (ProductID, ProductName, Price)
VALUES (1,'Laptop',1200)
COMMIT;
Rollback an Implicit Transaction:
UPDATE Products SET Price = 1300 WHERE ProductID = 1
ROLLBACK;
Do nothing and it will keep an open transaction:
DELETE FROM Products WHERE ProductID = 1

Since you have not COMMIT or ROLLBACK this statement, it will keep this as open transaction.
Reference Screen Shot:
What is SQL Server Transaction

Explicit transactions:

In SQL Server, an Explicit Transaction is a transaction that you start manually using BEGIN TRANSACTION and end using COMMIT or ROLLBACK. This gives you full control over when the transaction starts and ends.

Explicit Transaction with COMMIT:
BEGIN TRANSACTION

INSERT INTO Products (ProductID, ProductName, Price)
VALUES (1,'Laptop',1200)

UPDATE Products SET Price = 1300 WHERE ProductID = 1

DELETE FROM Products WHERE ProductID = 1

COMMIT;

This will save all the results of INSERT, UPDATE and DELETE. If you replace COMMIT with ROLLBACK command, it will save nothing.

Batch-scoped transactions:

Applicable only to Multiple Active Result Sets (MARS), when a Transact-SQL explicit or implicit transaction starts in a MARS session, SQL Server treats it as a batch-scoped transaction. If the transaction is not committed or rolled back before the batch finishes, SQL Server automatically rolls it back.

You may also like to go through the below articles:

What is SQL Server

Understanding SQL Server Versions and Editions

The Complete Guide to SQL Server System Databases

{ Add a Comment }

What Does SQL DBA Actually Do?

Hi There! Whether you are a student, aspiring data professional, or thinking to transform to a DBA role and wondering “What Does SQL DBA Actually Do”, you are not alone. The title Database Administrator (DBA) might sound mysterious, but it’s one of the most critical roles in IT .

What Does SQL DBA Actually Do

This guide breaks down what a SQL Server DBA really does, the tools they use and additional responsibilities as they grow in their carrier.

There are basically two type of DBA jobs in the IT industry.

Application DBA/Logical DBA — Works closely with Application Developers. Designs schema, tables, indexes, and relationships according to application needs. Require Strong SQL coding skills.

Production/Infrastructure DBA — Focuses on database operations, uptime, performance, and recovery in the production environment. Demands Strong system administration and database configuration skills.

To give you one real time scenario to understand a basic difference between the two:

When a Query is running slow, Production DBA will identify the exact slow running query and provide the query(execution) plan to App DBA. App DBA will analyze the plan and fix the query.

In some organizations the responsibilities may overlap depending upon the job requirement.

Key Responsibilities of a SQL Server DBA:

Area Application DBA Production DBA
Database Backup & Recovery Is not Responsible Ensure all database servers are backed up in alignment with Recovery Point Objective (RPO) and regularly
tests restores of the backup files to make sure Recovery Time Objective (RTO) can be met.
Database Design Designs schema, tables, indexes, and relationships according to application needs. Ensures design aligns with operational standards and performance requirements.
SQL & Performance Tunes SQL queries, stored procedures, and application-side performance. Tunes system-level performance—memory, I/O, CPU, and indexing strategies.
Development Support Works with developers during development and testing phases. Supports deployments, migrations, and production rollouts.
Maintenance Less involved in backups and system maintenance. Handles patching, upgrades, Index maintenance and monitoring.
Security & Access Defines roles and privileges for applications. Manages overall database security, auditing, and compliance.
Troubleshooting Fixes app-specific data or query issues. Diagnoses system-level issues, server crashes, corruption, and performance bottlenecks.

Essential Tools/Functionalities Every SQL Server DBA Should Know:

Tool Description
SQL Server Management Studio (SSMS) Core interface for managing SQL databases.
Azure Data Studio Modern, cross-platform alternative with notebooks.
SQL Server Extended Events For tracing and debugging query performance.
Dynamic Management Views (DMVs) Provides insight into SQL Server to assist with troubleshooting and monitoring.
SQL Agent Service Automates scheduled jobs and maintenance tasks.
PowerShell / dbatools For automating DBA workflows.
Query Store Simplifies the process of identifying and resolving Query performance issues by providing insights into query plan.
Monitoring Tool like Redgate SQL Toolbelt Professional suite for monitoring, comparison, and deployments.

Now as you grow in your DBA carrier, your role will not be limited to only technical work. I have tried to put few more critical responsibilities based on my experience.

Automation: Analyze repetitive tasks and check what and how far a task can be automated. Work with automation team, provide suggestions etc.

Incident reduction: Senior DBA/leads will work on analyzing ticket dumps and work on reducing repetitive incidents on same SQL Instance.

RCA (Root Cause Analysis) : On P1/P2 incidents, Leads/SME (Subject Matter Specialist) will work with Problem Management Team to document the RCA.

SOP (Standard Operating Procedure): You’ll be responsible for preparing SOPs for junior resources or on particular process to be followed for repetitive activities. e.g. Steps to Follow from database end during OS patching, Database Decommission, SQL Instance Provision/Decommission etc.

Customer Escalations: Handling customer escalations will be one of your prime responsibilities.

Mentoring Junior DBAs: Guiding Junior DBAs and providing necessary technical/process related trainings. Making proper plan for newly onboarded team member to bring him/her up to the speed etc.

Planning Major Activities: You will need to prepare Runbooks for major activities like DR (Disaster Recovery Drill), SQL Server upgrades etc.

Hope now you have got an overall idea about what Does a SQL DBA Actually Do. Please feel free to comment in case you want me to add any further responsivities.

You can also refer the following links:

What is SQL Server

Understanding SQL Server Versions and Editions

The Complete Guide to SQL Server System Databases

 

 

{ Add a Comment }

The Complete Guide to SQL Server System Databases

Hey there! this is the 3rd article in the category “SQL Server DBA Tutorial: Beginners to Pro”. Here we are going to discuss about SQL Server System databases. The prior article in this series is Understanding SQL Server Versions and Editions

Introduction

To begin with, when you install SQL Server, it creates the system databases automatically. Understanding the role of system databases is crucial. Here we are going to explore everything you need to know about SQL Server system databases,
including their purpose, components, and best practices for managing them.

Table of Contents

  • What are System Databases?
  • Importance of System Databases
  • Understanding the Core System Databases
    • Master
    • Model
    • Msdb
    • Tempdb
    • Resource Database
  • Distribution Database
  • Best Practices to maintain the System Databases

What are System Databases?

System databases are the backbone of a SQL Instance. Each of them play a critical role in the functioning of the database management system. These databases provide critical services to the server and user databases. Each system database has a specific purpose, and they work in harmony.

Importance of System Databases

System databases contain crucial information about the SQL Server instance, including server-level configurations, system objects, and metadata. Understanding the purpose and components of these databases is essential for efficient database management, backup and recovery, and overall system stability.

Understanding the Core System Databases

Now let’s take a deep dive into the core System Databases. The below image shows the system databases when you connect to a SQL Instance.

System Databases

Master Database:

Contains all the system-level information for an instance of SQL Server. The master database is the most important as it is necessary for starting the SQL Server.

This includes instance-wide metadata such as logon accounts, endpoints, linked servers, and system configuration settings.

Also, master is the database that records the existence of all other databases and the location of those database files and records the initialization information for SQL Server. Therefore, SQL Server cannot start if the master database is unavailable.

Model Database:

The model database is used as a template for other databases created in SQL Server, especially when creating user-defined databases.

The entire configuration of the model database, including database options, are copied to the newly created database.

Every time SQL Server is started, it creates tempdb from model hence if model is corrupted, SQL will not come up normally.

Msdb Database: 

The msdb database is used mainly by the SQL Server Agent, and also by other features like SQL Server Management Studio, Service Broker, and Database Mail.

SQL Server automatically maintains a complete online backup-and-restore history within tables in msdb.

Tempdb Database:

TempDB is a global resource that is accessed by all the users in the SQL Server Instance

This stores temporary user objects that you create like temporary tables (Global and Local), temporary variables, temporary stored procedures that hold intermediate results during the sorting or query processing, and cursors.

Internal objects like Work Tables, Work files. Intermediate sort results for operations such as creating or rebuilding indexes.

Version stores, which are collections of data pages that hold the data rows that support features for row versioning.

Resource database:

The Resource Database is a hidden, read-only database that contains system objects, system catalogs, and metadata.

SQL Server system objects, such as sys.objects, are physically persisted in the Resource database, but they logically appear in the sys schema of every database. The Resource database does not contain user data or user metadata.

Microsoft created this database to allow very fast and safe upgrades. You can’t see the database by selecting from sys.databases or
executing sp_helpdb. It also won’t show up in the system databases tree in the Object Explorer pane
of SQL Server Management Studio, and it doesn’t appear in the drop-down list of databases from your query windows.

However, this database still needs disk space.
You can see the files in your default binn directory by using Microsoft Windows Explorer. For Example: C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\Binn

mssqlsystemresource.mdf,  and mssqlsystemresource.ldf files.

To explain refer the below screen shot:

System Databases

Distribution Database:

SQL Server creates this database when you configure Replication on SQL Server and you choose the SQL Instance to act as a distributor. It stores replication metadata, history, and the transactions to replicate to other servers.

Best practices to Maintain System Databases:

Create a robust backup strategy for Regular backups of system databases.

Make proper SOP for restoring the system databases so that it is easier in the event of a disaster. Not to mention knowing how to restore system databases is essential for recovering your SQL Server instance.

Managing the size of system databases, especially TempDB, can significantly impact server performance. Hence proper Sizing and Growth Planning is very essential.

Equally important is monitoring solutions to keep an eye on the health and performance of your system databases.

In addition Consider placing system databases on separate disks to prevent contention and optimize performance.

{ Add a Comment }

Understanding SQL Server Versions and Editions

SQL Server Versions and Editions

Microsoft SQL Server is a popular database management system. It comes in different versions and editions, each with its own set of features and capabilities. Understanding the differences between SQL Server versions and editions can help you choose the right one for your organization’s needs. Here we’ll build the basic understanding of the terms and will not go into details.

This is second article in the SQL Server DBA Tutorial: Beginners to Pro series. You may want to read the first article What is SQL Server

SQL Server Versions

SQL Server continuously improves its features and capabilities by releasing different versions building upon the previous ones. The current version of SQL Server is SQL Server 2022, but previous versions, such as SQL Server 2019, 2017, 2016 and 2014 still widely used. Each version of SQL Server includes new features and enhancements that can improve performance and scalability.

There are other EOL (End Of Life) versions such as SQL Server 2012, 2008R2, 2008, 2005 and 2000.

As an illustration if I have to brief you about few of SQL Sever Versions:

SQL Server 2022:

The newest SQL Server version released on 16th November 2022. This is the most Azure-enabled version yet, building on its tradition of performance, security, and availability.

SQL Server 2019:

Microsoft launched SQL Server 2019 on November 4, 2019. Introduces Big Data Clusters for SQL Server. This lets you analyze your data on-fly using T-SQL and spark.

SQL Server 2017:

Launched on October 2, 2017. First time, Microsoft made the SQL Server compatible with Linux.

SQL Server 2016:

Introduced Always Encrypted in SQL 2016 in all editions. Prior to SQL Server 2016 (13.x) SP1, Always Encrypted was limited to the Enterprise Edition only.

SQL Server 2014:

Released to manufacturing on March 18, 2014, and released to the general public on April 1, 2014.

In essence, with each version Microsoft comes up with enhanced and robust features.

SQL Server Editions

SQL Server is also available in different editions, each with a specific set of features and capabilities. The most common editions of SQL Server are:

Standard Edition: This edition is suitable for small to medium-sized businesses and includes features such as basic backup and recovery, basic security, and basic reporting.

Enterprise Edition: Top-end edition with a full feature set. Basically large enterprises use this edition and includes advanced features such as advanced security, advanced data warehousing, and advanced reporting.

Express Edition: This edition is a free, entry-level version of SQL Server that is ideal for small to medium-sized applications and includes basic features such as basic backup and recovery, basic security, and basic reporting.

Web Edition: This edition provides basic features like basic backup and recovery, basic security, and basic reporting for web-based applications

Developer Edition: This is similar to Enterprise. Another free edition but only for development and testing environments.

Compact Edition: Microsoft SQL Server Compact 4.0 is a free, embedded database that software developers can use for building ASP.NET websites and Windows desktop applications.. The maximum size of the database is 4 GB.

Evaluation: The SQL Server Evaluation Edition is a great way to get a fully functional and free instance of SQL Server for learning and developing solutions. This edition has a built-in expiry of 6 months from the time that you install it.

Business Intelligence: Data warehousing and business intelligence applications uses this edition and includes advanced BI capabilities such as reporting and analysis services.

 Keep in mind that the feature set and limits of the different editions may change with new releases of SQL Server, so it’s best to check the specific product documentation for the most up-to-date information.

Conclusion

In Summary choosing the right SQL Server versions and editions can be challenging, especially when you consider the different features and capabilities that each version and edition provides. Thus by understanding the differences between SQL Server versions and editions, you can make an informed decision.

Hence it is important to keep in mind that the version and edition of SQL Server you choose will depend on the size of your organization, the type of data you need to store, and the features and capabilities you need to support your organization’s specific use cases.

{ Add a Comment }

What is SQL Server

Introduction to SQL Server

In this blog post, we will discuss What is SQL Server. You will learn its features, and how it manages and stores large amounts of data for organizations of all sizes. This is the first article in SQL Server DBA Tutorial: Beginners to Pro category.

What is SQL Server?

First thing to know is Microsoft Developed SQL Server and hence it is also called as MS SQL Server. It is a powerful RDBMS that allows organizations to store, manage, and retrieve data. Moreover SQL Server can handle both structured and unstructured data and work with a wide range of data types. SQL Server manages data for a variety of applications, including enterprise resource planning (ERP), customer relationship management (CRM), and data warehousing.

Features of SQL Server

SQL Server offers a wide range of features that make it a powerful tool for managing and storing data. Some of the key features of SQL Server include:

  • Scalability: SQL Server can scale up or down to meet the needs of any organization, from small businesses to large enterprises.
  • High availability: SQL Server can be configured for high availability to ensure that data is always available and that there is minimal downtime.
  • Security: SQL Server includes a variety of security features to help protect data from unauthorized access.
  • Business Intelligence: SQL Server equips a variety of business intelligence (BI) tools that allows users to analyze and visualize data.
  • Cloud-ready: SQL Server’s availability in both on-premise and cloud-based versions makes it easy to deploy and manage in any environment.

SQL Server usage

SQL Server can manage and store data for a wide range of applications. Some of the most common uses of SQL Server include:

  • Data warehousing: SQL Server stores and manages data for data warehousing applications. This allows organizations to analyze and report on large amounts of data.
  • Business Intelligence: SQL Server also stores and manages data for BI applications. This allows organizations to analyze and visualize data for decision making.
  • Website and Application: Many websites and applications uses SQL Server serves as a backend, it also stores and retrieves data quickly and efficiently.

In conclusion, SQL Server is a powerful RDBMS which stores, manages, and retrieves data for a wide range of applications. As can be seen, with its wide range of features and scalability, it is well-suited for organizations of all sizes. If you’re looking for a powerful and flexible tool for managing your data, SQL Server is a great option to consider.

Checkout the following articles in the same category and all the best in your SQL DBA journery:

Understanding SQL Server Versions and Editions

The Complete Guide to SQL Server System Databases

What Does SQL DBA Actually Do?

What Is SQL Server Transaction: Complete Beginner’s Guide

 

{ Add a Comment }