Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion GroupsVB SyntaxEnterprise DevelopmentDatabase AccessControlsCOMWin APICrystal ReportDeploymentGeneralGeneral 2
Related Topics
VB.NET / ASP.NETMS SQL ServerMS AccessOther Database ProductsMore Topics ...

VB Forum / COM / October 2007



Tip: Looking for answers? Try searching our database.

Thread-safe Collection

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Tony Proctor - 05 Sep 2007 16:38 GMT
Anyone know of a thread-safe alternative to a Collection or Dictionary that
could be used in VB?

The existing Collection/Dictionary work OK in STA threaded environments,
except that their storage is per thread.

What I'm looking for is something that can be used by multiple threads,
without cross-thread marshalling, and maintains only a single dictionary in
memory.

   Tony Proctor
Schmidt - 10 Sep 2007 04:17 GMT
> Anyone know of a thread-safe alternative to a Collection
> or Dictionary that could be used in VB?
[quoted text clipped - 5 lines]
> multiple threads, without cross-thread marshalling, and
> maintains only a single dictionary in memory.

What datatype(s) would you want to store inside such
a collection?
I mean, if you want to allow Objects to be stored too,
then accessing such an object over the threadsafe-
collection would not be possible without marshalling.

But a Dictionary/Collection without that requirement
(storing objects inside) would be possible (without
marshalling, only using critical sections).

Olaf
Tony Proctor - 10 Sep 2007 11:19 GMT
They would be 'simple data types' Olaf, i.e mainly text and numbers. Hence,
no Objects, and so no apartment violations :-)

My problem is basically with the use of a Dictionary to cache certain
information in a web server environment. The standard Collection or
Dictionary accessible to VB will only cache the information for the
associated apartment. This obviously limits to efficiency, but worse still
it causes memory bloat. The memory used by the cache is multiplied by the
number of ASP threads (AspProcessorThreadMax * number of processors) which
could easily a factor of 200 (e.g. with 4 dual-core processors). Since IIS
struggles to even make a full 2Gb available to the w3wp worker process
(mainly due to fragmentation issues) then the cache wastes a precious
resource.

   Tony Proctor

> > Anyone know of a thread-safe alternative to a Collection
> > or Dictionary that could be used in VB?
[quoted text clipped - 17 lines]
>
> Olaf
Schmidt - 10 Sep 2007 16:04 GMT
> They would be 'simple data types' Olaf, i.e mainly text and
> numbers. Hence, no Objects, and so no apartment violations :-)
Ok, that would be important, to meet your requirement:
"no marshaling please". :-)

But why would you want to avoid marshaled calls (e.g.
against a dictionary inside a "GetObject-Singleton").
What is your "request-frequency" from your Web-Clients
against such a "marshaled accessible Singleton-Collection".
On modern machines/CPUs you can calculate with at
least ca. 4000 Calls per second (e.g. search an item by
Key) against such an construct.

Background is, that you would have to implement
a non-marshaled solution with higher coding-effort,
meaning you would use STA-local (lightweight and
selfwritten) Collection-Wrappers, wich would share
e.g. a process-global heap-handle and work inside
their STAs using Critical-Sections as soon as you
want to change the Data-Content "behind" the global
Heap-Object in a "Memory-Transaction". As said, this
would work much faster than only with 4000 keyed
item-requests per second, but would mean a greater
implementation-effort than "marshaled access to a global
reachable Singleton".

Olaf
Tony Proctor - 10 Sep 2007 18:09 GMT
Performance of the server code is critical, and any cross-thread marshalling
would kill performance. As I said earlier, there could easily be 200 ASP
threads handling thousands of concurrent users. The code is already highly
stressed due resource requirements of Analysis Services, and to a lesser
extent by ADODB and MSXML

   Tony Proctor

> > They would be 'simple data types' Olaf, i.e mainly text and
> > numbers. Hence, no Objects, and so no apartment violations :-)
[quoted text clipped - 23 lines]
>
> Olaf
Schmidt - 11 Sep 2007 14:00 GMT
> Performance of the server code is critical, and any
> cross-thread marshalling would kill performance.
Not "per se" - you could calculate the effect on performance
relative reliable.

> As I said earlier, there could easily be 200 ASP
> threads handling thousands of concurrent users.
Ok - assuming we have 5000 users, each triggering
a "server-action" wich provokes a "marshalled Dictionary-
Lookup" every 10 seconds on average.
Then we have ca. 500 marshalled Lookups per second
on the serverside (on average).

As said, a modern single CPU reaches "Full-Load" with a
simple marshalled request (SendMessage-Calls going on
under the hood) on ca. 4000 Calls per second.
If you have an 8way-server, then you cannot extrapolate
this value lineary, but you can probably at least double this
value (let's say to 10000 marshalled Calls per second with
an average CPU-Load over all 8 CPUs of ca 25-35%).

Now for the scenario above: 5000 concurrent users,
each provoking a marshaled Dictionary-Lookup every 10
seconds, the caused CPU-Load would be:
for ca. 500 marshaled lookups/sec = 1.25-1.75%
if 10000 marshaled lookups/sec (worstcase) = 25-35%

So I would write a small test-scenario and check first,
how it performs in your environment with marshalled calls.

Olaf
Tony Proctor - 13 Sep 2007 17:29 GMT
I thought about this a little more Olaf. You cannot guarantee the
performance of these objects since it depends to some extent on how their
memory is spread out, and how you access it. Performance isn't bad using
only hashing (up to a limit) but indexing is very bad.

Also, the type of singleton I would have to use would have to be
process-specific, not machine-wide, and not specific only to the class being
instantiated. With a normal VB project you'd do this via static data in a
Module, but there is no shared data in a multithreaded VB context.

   Tony Proctor

> > Performance of the server code is critical, and any
> > cross-thread marshalling would kill performance.
[quoted text clipped - 27 lines]
>
> Olaf
Schmidt - 14 Sep 2007 14:07 GMT
> I thought about this a little more Olaf.
> You cannot guarantee the performance of these objects
> since it depends to some extent on how their
> memory is spread out, and how you access it.
What do you mean with "these Objects".
If you pin a singleton somewhere, then this singleton-
class implements only one (private) Collection internally.
And the marshaled performance I've measured is
of course only related to access-attempts to the
*interface* of this singleton-object.
The additional time needed for accessing the internal
dictionary-based Cache-Object would have to be
respected/added of course - but these "extra-costs" would
you see even in an unmarshaled approach, wich uses
CriticalSections instead of the builtin "serialized access to
a shared resource", wich OLE-Marshalling offers "for free".

A different story would be, if your shared resource is more
static regarding its content and should allow parallel reading
from multiple STAs - but that's what Databases are built for.
If you want to have fast parallel reads from multiple STAs,
and secure, serialized writes you can use e.g. SQLite, wich
has low memory-footprint and offers optimized search-strategies
on indexed Data and even full-text-table-searches.

In my thinking about a fast, singleton-internal Dictionary-Object
I would lock even the Read-Direction for security-reasons,
because a parallel Adding, whilst another STA reads Data
could cause inpredictable results.

You would have to decide/measure, if a fast, but serialized
Dictionary-Access performs better, than access against
a DataBase-Connection, wich could process Read-Requests
in parallel from multiple STAs, but has somewhat more
overhead compared with a "Singleton-Dictionary".

> Performance isn't bad using
> only hashing (up to a limit) but indexing is very bad.
You could use e.g. my much (regarding indexed access)
faster Sorted-Dictionary implementation as the private
Object inside your Singleton-Class.

> Also, the type of singleton I would have to use would
> have to be process-specific, not machine-wide, and
> not specific only to the class being instantiated.
You mean, the shared singleton-object would have to
be hosted inside the IIS-Process. IIRC you can "pin"
such an singleton-object using ASP in the global
Application-Cache:
Application.Lock
Set Application("YourKey") = CreateObject("My.SingletonCache")
Application.UnLock

or better yet in
Application_OnStart of the Global.asa
Cleanup/Shutdown then in:
Application_OnEnd

retrieving the marshaled Singleton:
Set MySingletonCache = Application("YourKey")

Since "transactions" only work on Call-Basis with
Ole-Marshalling you should take that into respect
and avoid multiple interface-calls.
E.g. something like this should be avoided:
If Not MySingletonCache.EntryExists(Key) Then
   MySingletonCache.AddEntry  Key, Entry
End If
Or even worse: For Each-Enumerations...
All those multiple calls have to be encapsulated internally.

Instead you will have to handle it in one single call:
MySingletonCache.AddEntryIfNotExists Key, Entry
Only this way you can be sure, that the Marshaller
blocks other STAs - meaning: you have Transaction-safety -
but only at "Single-Call-Level".

> With a normal VB project you'd do this via static data in a
> Module, but there is no shared data in a multithreaded VB
> context.
You can share memory relative easy between threaded
STAs in VB using SafeArray-Pointers.
Of course you have to regulate the access to such shared
memory-Resources with your own Locking-mechanisms
using CriticalSections then.
If you want, I can write an example for you, wich shares
a SortedDictionary-Instance between multiple STAs
in a standard VB-Exe, but regarding your IIS-host I'd
recommend the *marshaled* Singleton-approach
(or alternatively a lightweight DB-Engine) anyway.

Olaf
Tony Proctor - 15 Sep 2007 13:24 GMT
Thanks for all this Olaf.

You're right about the Application object. I hadn't even considered that
because all the Microsoft documentation is adamant that it should not be
used for storing STA objects because of cross-thread issues, and the
supposed serious impact on the performance of the web server.

In my most important case, the dictionary is a global read-only cache.
Obviously it would have to be populated by one of the threads during
startup, but after that it would only be supporting multiple readers. Hence,
it would still need read-locks and write-locks to be safe but the write-lock
contention would be almost non-existent.

Ideally, I wanted the dictionary items in a common memory store (to avoid
the VM size getting multiplied per thread), but still allow multiple
concurrent readers for maximum performance

I'll investigate your suggestions as soon as I can

   Tony Proctor

> > I thought about this a little more Olaf.
> > You cannot guarantee the performance of these objects
[quoted text clipped - 88 lines]
>
> Olaf
Schmidt - 15 Sep 2007 17:28 GMT
> You're right about the Application object. I hadn't even
> considered that because all the Microsoft documentation
> is adamant that it should not be used for storing STA objects
> because of cross-thread issues, and the supposed serious
> impact on the performance of the web server.
Yep, but that's more with regards to "inflationary usage every
now and then" - but to pin a singleton there, the Application-
object seems to be applicable.

> In my most important case, the dictionary is a global
> read-only cache.
Ah, Ok.

> Obviously it would have to be populated by one of the threads
> during startup,
As said, the population can very well sit inside the Singletons-
Class_Initialize - and this Singleton-Class itself could be
instantiated and "pinned" into the Application-object inside
the Global.asa - Application_OnStart.

> but after that it would only be supporting multiple readers.
> Hence, it would still need read-locks and write-locks to be
> safe but the write-lock contention would be almost non-existent.
If the Singleton-Content is really static (not changed during
the Application-Livetime), you don't even need the read-locks.

The Singleton-Class, wich is keeped alive inside the
Application-Object would only have to offer a fast way
for retrieving the appropriate Array-Pointers.
E.g. you could implement that, if this Singleton creates
a hidden Form inside its Class_Initialize with a predefined
and unique Caption. After populating two (private member-)
arrays inside your singleton (one presorted String-Array for
the String-Keys and one "index-matching" Array, based on
a defined Private Type for the Content-Members), you
could create this invisible Helper-Form and simply set
two Properties on it (SetProp-API), wich contain the
startpointers for the two Arrays.

Your parallel Readers inside your STAs could then retrieve
these two pointers very fast using FindWindow-GetProp-
APIs, and set these Pointers into safeArray-Structures
inside a lightweight-Dictionary-Implementation-Class
as the CacheReader in each STA - all without any
Marshalling, allowing multiple and *parallel* Read-Access
against the so shared Arrays, hosted in the Singleton-Class
wich is kept alive in the Application-Object.
These lightweight Access-Classes would then be able,
to use the vitually spanned safeArrays for very fast indexed
access - and due to the presorted nature of the Keys
in the String-Array you could also achieve very fast
keyed-access using a simple Binary-Chop against
the virtually spanned KeyString-Array.

Olaf
Tony Proctor - 15 Sep 2007 19:26 GMT
I agree with most of what you said Olaf, except the 'hidden Form'. I don't
believe that's really necessary, and a project with 'Retain in Memory' &
'Unattended execution' cannot have any UI elements in it anyway.

   Tony Proctor

> > You're right about the Application object. I hadn't even
> > considered that because all the Microsoft documentation
[quoted text clipped - 51 lines]
>
> Olaf
Schmidt - 15 Sep 2007 19:59 GMT
> I agree with most of what you said Olaf, except the
> 'hidden Form'. I don't believe that's really necessary,
> and a project with 'Retain in Memory' & 'Unattended
> execution' cannot have any UI elements in it anyway.
Thought about the CreateWindowEx-API instead of
"VBRuntime-Controlled Forms". This should work -
even in 'Unattended execution-mode' - the whole OLE-
Marshalling is using hidden forms under the hood - and
this works well, even without any loaded desktop
in "pure service-mode".

But you are right, there are of course additional ways, to
make two pointer-values (and maybe the Array-Count too)
available in a fast way (without any marshalled calls) to STAs,
wich need them.

Regards,

Olaf
Tony Proctor - 18 Oct 2007 21:15 GMT
This all breaks down Olaf because you cannot store STA objects in the ASP
Application object

   Tony Proctor

> > You're right about the Application object. I hadn't even
> > considered that because all the Microsoft documentation
[quoted text clipped - 51 lines]
>
> Olaf
Schmidt - 19 Oct 2007 01:46 GMT
> This all breaks down Olaf because you cannot store STA
> objects in the ASP Application object
Ah Ok - long time ago, since I used the IIS and its
environment (in some older version this has worked).

Ok, then try this approach:
(just tested this with an IIS, wich was installed and
updated with VS2005-Express - and it works well,
no global.asa, no special initializing)

Simply use an *.asp-script (called from a browser) for testing.
<% Server.CreateObject("Your.ProgID").DoIt Response %>

You can adjust the UserEntries in the Demo and watch
the Memory-consumption of dllhost.exe in the TaskManager.

'And this goes into a Public Class in an AX-Dll
Option Explicit

Private Type TGlob 'always use fixed length strings inside the Types
 Initiated As String * 10
 LastThreadID As Long
 LastAccess As Date
 MaxArrEntryCount As Long
 CurArrEntryCount As Long
 '...
End Type

Private Type TGlobArray 'always use fixed length strings inside the Types
 ID As Long
 UserName As String * 128
 '...
End Type

Private Declare Sub RtlMoveMemory Lib "kernel32" _
 (Dst As Any, Src As Any, ByVal bc&)
Private Declare Function ArrPtr& Lib "msvbvm60" Alias "VarPtr" _
 (Arr() As Any)
Private Declare Function CloseHandle& Lib "kernel32" _
 (ByVal hObj&)
Private Declare Function CreateFileMappingA& Lib "kernel32" _
 (ByVal hFile&, Attr As Any, ByVal Protect&, ByVal SizeHigh&, _
  ByVal SizeLow&, ByVal MapName$)
Private Declare Function MapViewOfFile& Lib "kernel32" _
 (ByVal hFile&, ByVal DesAccess&, ByVal OffsHigh&, ByVal OffsLow&, _
  ByVal NumberOfBytesToMap&)
Private Declare Function UnmapViewOfFile& Lib "kernel32" _
 (ByVal lpBaseAddress As Long)

Private Const MapKey$ = "MyMapKey", FILE_MAP_ALL_ACCESS& = &HF001F

Private Const MaxUsers& = 100

Private Glob() As TGlob, saGlob&(5)
Private Users() As TGlobArray, saUsers&(5)
Private BaseAddr&, hFile&, Initiator As Boolean

Public Sub DoIt(Response)
Dim i As Long, NewUserIdx As Long

 If BaseAddr = 0 Then Exit Sub

 With Glob(0)
   Response.Write "<PRE>"
   Response.Write "Global BaseData:<BR>"
   Response.Write "  LastThreadID: " & .LastThreadID & "<BR>"
   Response.Write "  CurrentThreadID: " & App.ThreadID & "<BR>"
   Response.Write "  LastAccess: " & .LastAccess & "<BR>"
   Response.Write "  CurrentAccess: " & Now & "<BR><BR>"

   .LastThreadID = App.ThreadID
   .LastAccess = Now

   Response.Write "We have enough memory for: " & _
                  .MaxArrEntryCount & " Users<BR><BR>"
   'lets Add another User
   NewUserIdx = .CurArrEntryCount
   If NewUserIdx < .MaxArrEntryCount Then
     .CurArrEntryCount = NewUserIdx + 1
     'Now we have time, to fill in the members at the NewUserIdx
     With Users(NewUserIdx)
       .ID = NewUserIdx + 1
       .UserName = "UserName_" & .ID & vbNullChar
     End With
     NewUserIdx = NewUserIdx + 1
   End If

   Response.Write "Current UserList has: " & NewUserIdx & " Users<BR>"
   For i = 0 To NewUserIdx - 1
     With Users(i)
       Response.Write "  UserID: " & .ID & _
                      ", UserName: " & F2S(.UserName) & "<BR>"
     End With
   Next i
   Response.Write "</PRE>"
 End With
End Sub

Private Sub Class_Initialize()
Dim GlobDummy As TGlob, MapBaseSize&
Dim GlobArrayDummy As TGlobArray, MapUserSize&
 MapBaseSize = LenB(GlobDummy) 'always use LenB
 MapUserSize = LenB(GlobArrayDummy) * MaxUsers 'always use LenB

 'Initialize safearray-descriptor
 saGlob(0) = 1 'Dimension
 saGlob(1) = LenB(GlobDummy) 'size of Member
 saGlob(4) = 1 'Number of Array-Elements
 'array-binding (bind descriptor to the Glob-Arr)
 RtlMoveMemory ByVal ArrPtr(Glob), VarPtr(saGlob(0)), 4

 'now the same for the User-Array
 saUsers(0) = 1 'Dimension
 saUsers(1) = LenB(GlobArrayDummy) 'size of Member
 saUsers(4) = MaxUsers 'Number of Array-Elements
 'array-binding (bind descriptor to the Users-Arr)
 RtlMoveMemory ByVal ArrPtr(Users), VarPtr(saUsers(0)), 4

 'do the Mapping inside the SwapFile
 hFile = CreateFileMappingA(-1, ByVal 0&, 4, 0&, _
         MapBaseSize + MapUserSize, MapKey)
 If hFile = 0 Then Exit Sub

 BaseAddr = MapViewOfFile(hFile, FILE_MAP_ALL_ACCESS&, 0&, 0&, 0&)
 If BaseAddr = 0 Then Exit Sub

 saGlob(3) = BaseAddr 'set the DataPointer to the mapped area
 If F2S(Glob(0).Initiated) <> "Initiated" Then
   Initiator = True
   Glob(0).Initiated = "Initiated" & vbNullChar
   Glob(0).MaxArrEntryCount = MaxUsers
 End If

 'finally we set the Users-DataPointer behind the Glob-Data
 saUsers(3) = BaseAddr + MapBaseSize
End Sub

Private Sub Class_Terminate()
 RtlMoveMemory ByVal ArrPtr(Glob), 0&, 4 'reset array-binding
 RtlMoveMemory ByVal ArrPtr(Users), 0&, 4 'reset array-binding

 If Not Initiator Then 'if we are not the initiator, let's cleanup here
   If BaseAddr Then UnmapViewOfFile BaseAddr: BaseAddr = 0
   If hFile Then CloseHandle hFile: hFile = 0
 End If
End Sub

Private Function F2S$(FLStr$) 'Helper to convert fixed to BSTR
Dim Pos&
 Pos = InStr(FLStr, vbNullChar)
 If Pos Then F2S = Left$(FLStr, Pos - 1)
End Function

Olaf
Tony Proctor - 19 Oct 2007 17:25 GMT
My Dictionaries (there are several) could have 20k-30k items so a binary
chop wouldn't be quick enough Olaf.

I've tried using an Application Singleton created via GLOBAL.ASA but having
other issues there (see more recent thread in this group). If that fails, is
there a way of using the ROT to hold a reference to an object within just a
specific process? What I want to avoid is the risk of multiple web
applications that use the same DLL accidentally sharing the same instance of
my singleton. In other words, can the ROT be used to implement a
"process-wide singleton" rather than a "machine-wide singleton"?

   Tony Proctor

> > This all breaks down Olaf because you cannot store STA
> > objects in the ASP Application object
[quoted text clipped - 150 lines]
>
> Olaf
Schmidt - 19 Oct 2007 19:12 GMT
> My Dictionaries (there are several) could have 20k-30k items
>  so a binary chop wouldn't be quick enough Olaf.
For 20-30k items the bin-search would only need to do ca.
12 string-compares on average to find a record over the
appropriate String-Key.

> I've tried using an Application Singleton created via GLOBAL.ASA
> but having other issues there (see more recent thread in this group).
Yea, seen that - found especially interesting, that the "outside WebRoot"
Dlls caused less problems than the "insiders" - one would expect that
exactly the other way round.

> If that fails, is there a way of using the ROT to hold a reference
> to an object within just a specific process?
Not really, but if you google for one of my ROT-postings here
in this group (ca. two years ago), then you will find a routine,
wich is able to place and retrieve a VB-Class in/from the ROT
over a free definable FileMoniker(String) - this way you could
achieve different instances of the same Cache-Class for different
processes (wich simply would have to know "their key").

> In other words, can the ROT be used to implement a
> "process-wide singleton" rather than a "machine-wide singleton"?
As said, not directly, but over a special "ROT-Key".
The problem with the ROT-Objects is, that you will have to
see, if the IIS allows their instantiation/registration in the ROT
and that there are a few problems with them regarding
"pure service mode" - meaning if the OS is not bootet into
a Window-Station or -Desktop.
But the code I've mentioned above does to try to take
that into respect IIRC.

With my just posted example here, there is only coding to
do - no surprises possible (if coded correctly).
The singleton-approach (be it in the ROT or the
global.asa) could offer some additional fun, even if you've
managed the "singleton-instantiation-problems".

Its something like a "pronounced traffic jam" where you
know about a definitely "jam-free", alternative way, wich
on the other hand would be some miles longer to drive. ;-)

Olaf
Tony Proctor - 19 Oct 2007 19:30 GMT
This sounds promising. I've used the ROT in other contexts Olaf, but I
always use the classid as the key - I didn't know if was freely definable.
Do you have a reference for that post?

I didn't understand the reference about booting into a window station. The
other contexts I've used the ROT are purely server-side (no UI), just like
this new ASP requirement.

   Tony Proctor

> > My Dictionaries (there are several) could have 20k-30k items
> >  so a binary chop wouldn't be quick enough Olaf.
[quoted text clipped - 39 lines]
>
> Olaf
Schmidt - 19 Oct 2007 20:53 GMT
> This sounds promising. I've used the ROT in other
> contexts Olaf, but I always use the classid as the key -
> I didn't know if was freely definable.
> Do you have a reference for that post?
Just search inside google.groups with these keywords:
rot vb sss matthew

> I didn't understand the reference about booting into a
> window station.
Just follow the thread (you were taking part BTW)... ;-)

Olaf
Tony Proctor - 20 Oct 2007 10:50 GMT
Ah, that thread! Yes, at the moment I only use RegisterActiveObject for
working with the ROT since I don't need any extra typelibs. However, the
downside is that it doesn't offer file-moniker binding.

Since a file-moniker must appear in the registry then I have to consider the
possibility of accidentally leaving artefacts in there since an ASP app
doesn't really have a "close down" (Session-end and Application-end are
useless for anything like that). This means I cannot use a transient
differentiator like the process ID. I probably need something related to the
relevant virtual directory, which in turn is configured by customers, not
myself.

   Tony Proctor

> > This sounds promising. I've used the ROT in other
> > contexts Olaf, but I always use the classid as the key -
[quoted text clipped - 8 lines]
>
> Olaf
Tony Proctor - 10 Sep 2007 19:05 GMT
...and, I forgot to say: I was hoping to find one off-the-shelf  :-)

   Tony Proctor

> > They would be 'simple data types' Olaf, i.e mainly text and
> > numbers. Hence, no Objects, and so no apartment violations :-)
[quoted text clipped - 23 lines]
>
> Olaf
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.