Services
Superbase
Networking - Oplocks Explained
Introduction
This document was posted to the comp.lang.clipper USENET newsgroup
on the 12th of December 2002. It is a reproduction of a paper that
was distributed at a recent Xbase++ DevCon. This is reproduced below
in its entirety, in order to ensure that the link does not vanish.
The link to the original posting can be found here.
This document is not produced with permission, but we are currently
seeking the author in order to correctly attribute the contents
of this page. The article is virtually as it was posted, minor spelling
corrections and markup to make it consistent with our site have
been applied.
Opportunistic Locking
This article is about opportunistic locking, a little known protocol
of the LAN Manager of MS Windows NT/2000 and XP designed to increase
performance of network application — in theory!
However this protocol imposes a couple of problems for Xbase++/Clipper
and even MS Access or Excel applications running in a shared network
environment.
In this article I would like to explain the basic idea of oplocks,
which problems they implicate for network applications and, last
but not least, various strategies how to increase performance and
reliability of your network application.
What Is Opportunistic Locking?
Opportunistic locks (oplocks) are a characteristic of the LAN
Manager networking protocol implemented in the 32-Bit Windows family
of operating system environments. Bascially, oplocks are guarantees
made by a server for a shared logical volume to its clients. These
guarantees inform the Client that a file's content will not be allowed
to be changed by the server, or if some change is imminent, the
client will be notified before the change is allowed to proceed.
So much about the theory, in practice the LAN Manager introduces
with oplocks an additional protocol on top of the well known standard
file-system level locking protocol. This is done for performance
reasons. Only with oplocks the server and its clients are allowed
to cache data (read ahead) or defer writes to a file (write behind).
In other words, oplocks are introduced to synchronize the data in
an environment of multiple workstations, each of which having their
own local cache, sharing a file via a file-server. Remember, any
workstation has its own cache
to increase performance when it comes to file I/O. Imagine a situation
where 2 workstations read data (into their local cache) and a single
workstation writes data back. As a matter of fact, one workstations
cache is out-of-date and holds
"old" data. With oplocks the local cache of workstation
2 gets informed about the fact of modification by other workstations
and therefore invalidates its local cache.
The problem with oplocks is that they:
- are designed for file-sharing with medium concurrency
- introduce a additional level of overhead
- block clients from performing their operations
immediately.
In practice the oplocks introduced by Microsoft are good enough
when it comes to sharing files like Word documents or Excel spreadsheets
in a networking environment. But they fail when it comes to heavy
concurrency in enviroments with file-based databases such as Xbase++,
Visual FoxPro and even MS-Access or VB applications with the Jet-Engine.
There are other strategies available, such as
the ones implemented with the Andrew's file-system which deal with
the problem of distributed caches in a network, but for whatever
reason Microsoft has decided to go their own route.
The reason for the problems the oplock approach has with file
based database systems lies in their specific characteristic how
they deal with the file-system. File based database systems:
- impose a heavy concurrency and high propability
of data changes anywhere in the file.
- handle concurrency issues on their own at the
semantical layer of the database. Not on the file-based level.
Remember they lock records, ensure that updates get not lost using
sophisticated implicit and explicit locking protocols.
- implement specialized local optimization strategies
to handle caching of data and invalidation of cached data. All
based on the record or index-page level.
- This is where the universal oplock solution with
Microsoft LAN Manager and file-based databases conflicts and can
serious hamper them in terms of performance and reliability. Even
deadlocks produced by the LAN Manager become possible with high
concurrency due to its restricted handling of so called worker
threads in the context of oplocks.
With Windows for Workgroups, IBM LanManager and Novell Netware
oplocks problems do not exist bec. these LAN operating-systems resolve
the problems differently or simply have not implemented a optimization
strategy such as the one introduced by the oplocks under Microsoft
LAN Manager.
Which Applications May Be Affected?
In general database applications but also office applications
when online collaboration is enabled are affected by the oplock
problem when using Windows NT/2000 and XP as the file server. Since
Windows NT 4.0 Service Pack 4, 5, 6, or 6a and Windows 2000 a default
oplock configuration which is critical to all types of applications
which run in a environment of heavy concurrency was introduced.
Not affected:
- Windows NT 4.0 with Service Packs prior of #4
- Windows ME/98/95
Affected:
- Windows NT 4.0 with Service Pack 4 or higher
- Windows 2000
- Windows XP
At best your network performance is sluggish,
at worst you encounter problems with an open file shared by the
LAN Manager such as:
- The file is in use
- The file is already open
- Unable to write to drive U:
- The network drive U: is no longer available
- Data loss
- Data corruption - file size inconsistency
How Microsoft Reacts To This Issue?
You can find various MS Knowledgebase articles about this issue.
MS has officially confirmed that the default oplock configuration
is a problem for "DOS" style database applications. However,
understanding the Microsoft terminology is critical here:
1. The term "older MS-DOS-based database applications"
refers to all file-based database applications. Because they perform
a so called heavy concurrency pattern.
2. MS has no strategic interest in resolving this problem, specifically
in the context of the MS-Access and Jet-DatabaseEngine users community.
Because now they can talk them into using MS-SQL Server instead
of resolving the performance and reliability problems.
The following MS Knowledgebase articles are of interest to the
entire subject:
Q219022
Improving Performance of MS-DOS Database Applications
Because Xbase++ database engines in the context of Clipper/Comix
and SIX interoperability behave against the file system such as
a MS-DOS database application this article is valid for DBF/NTX/CDX
drivers et al.
Q129202
Explanation of Opportunistic Locking on Windows NT
This article describes additional configuration parameters regarding
oplocks. The article is valid for NT/2000 and XP.
Q163525
Delay When Saving Word File to Windows NT 4.0 Server
Even MS Word and other Office applications may run into performance
problems with the oplock solution by Microsoft.
Q126026
Server Optimization in RFCB Caching
If you have encountered problems when opening files located on a
server. This article describes how to detect if the problem is related
to oplocks and the CachedOpenLimit configuration parameter.
Q249799
Slow Network Performance with Service Pack 4, 5, 6, or 6a
Valid for Xbase++/Clipper file-based database applications.
Q163525
Delay When Saving Word File to Windows NT 4.0 Server
FYI: Office 2000 has problems with oplocks too! You are not alone!
What To Do?
The simplest way to resolve problems related to oplocks is to
disable them on behalf of the LAN Manager server.
This is done by manipulating the following registry key:
\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\LanmanServer\Parameters
EnableOplocks REG_DWORD 0 or 1
Default: 1 (true)
Specifies whether the server allows clients to use oplocks on files.
Oplocks are a significant performance enhancement, but have the
potential to cause lost cached data on some networks, particularly
wide-area networks.
By default, the registry entry is 1 (oplock enabled), as a rule
of thumb you should set this key to 0 (disable oplock) when sharing
files with Xbase++/Clipper/FoxPro and MS-Access. We have not encountered
any performance drawbacks in real-world scenarios after having disabled
oplocks. However problems with Xbase++ file-based database applications
simple went away after re-configuration of the server.
Summary
Opportunistic locks where introduced by Microsoft to increase
performance of network applications when it comes to file sharing.
The strategy used significantly decreases performance of file-based
database applications. The oplock can easily and safely be disabled
via a registry-key setting change. After disabling of oplocks your
database application should show increased performance and higher
reliability. Other applications may but typically do not encounter
performance penalties. The only potential drawback of disabling
oplocks is a potential higher bandwidth usage when it comes to standard
file usage such as opening/closing/reading/writing office documents.
This page was last reviewed/modified on
.
|
Legal Notice
