Thread-safe Collection
|
|
Thread rating:  |
Tony Proctor - 05 Sep 2007 16:38 GMT Anyone know of a thread-safe alternative to a Collection or Dictionary that could be used in VB?
The existing Collection/Dictionary work OK in STA threaded environments, except that their storage is per thread.
What I'm looking for is something that can be used by multiple threads, without cross-thread marshalling, and maintains only a single dictionary in memory.
Tony Proctor
Schmidt - 10 Sep 2007 04:17 GMT > Anyone know of a thread-safe alternative to a Collection > or Dictionary that could be used in VB? [quoted text clipped - 5 lines] > multiple threads, without cross-thread marshalling, and > maintains only a single dictionary in memory. What datatype(s) would you want to store inside such a collection? I mean, if you want to allow Objects to be stored too, then accessing such an object over the threadsafe- collection would not be possible without marshalling.
But a Dictionary/Collection without that requirement (storing objects inside) would be possible (without marshalling, only using critical sections).
Olaf
Tony Proctor - 10 Sep 2007 11:19 GMT They would be 'simple data types' Olaf, i.e mainly text and numbers. Hence, no Objects, and so no apartment violations :-)
My problem is basically with the use of a Dictionary to cache certain information in a web server environment. The standard Collection or Dictionary accessible to VB will only cache the information for the associated apartment. This obviously limits to efficiency, but worse still it causes memory bloat. The memory used by the cache is multiplied by the number of ASP threads (AspProcessorThreadMax * number of processors) which could easily a factor of 200 (e.g. with 4 dual-core processors). Since IIS struggles to even make a full 2Gb available to the w3wp worker process (mainly due to fragmentation issues) then the cache wastes a precious resource.
Tony Proctor
> > Anyone know of a thread-safe alternative to a Collection > > or Dictionary that could be used in VB? [quoted text clipped - 17 lines] > > Olaf Schmidt - 10 Sep 2007 16:04 GMT > They would be 'simple data types' Olaf, i.e mainly text and > numbers. Hence, no Objects, and so no apartment violations :-) Ok, that would be important, to meet your requirement: "no marshaling please". :-)
But why would you want to avoid marshaled calls (e.g. against a dictionary inside a "GetObject-Singleton"). What is your "request-frequency" from your Web-Clients against such a "marshaled accessible Singleton-Collection". On modern machines/CPUs you can calculate with at least ca. 4000 Calls per second (e.g. search an item by Key) against such an construct.
Background is, that you would have to implement a non-marshaled solution with higher coding-effort, meaning you would use STA-local (lightweight and selfwritten) Collection-Wrappers, wich would share e.g. a process-global heap-handle and work inside their STAs using Critical-Sections as soon as you want to change the Data-Content "behind" the global Heap-Object in a "Memory-Transaction". As said, this would work much faster than only with 4000 keyed item-requests per second, but would mean a greater implementation-effort than "marshaled access to a global reachable Singleton".
Olaf
Tony Proctor - 10 Sep 2007 18:09 GMT Performance of the server code is critical, and any cross-thread marshalling would kill performance. As I said earlier, there could easily be 200 ASP threads handling thousands of concurrent users. The code is already highly stressed due resource requirements of Analysis Services, and to a lesser extent by ADODB and MSXML
Tony Proctor
> > They would be 'simple data types' Olaf, i.e mainly text and > > numbers. Hence, no Objects, and so no apartment violations :-) [quoted text clipped - 23 lines] > > Olaf Schmidt - 11 Sep 2007 14:00 GMT > Performance of the server code is critical, and any > cross-thread marshalling would kill performance. Not "per se" - you could calculate the effect on performance relative reliable.
> As I said earlier, there could easily be 200 ASP > threads handling thousands of concurrent users. Ok - assuming we have 5000 users, each triggering a "server-action" wich provokes a "marshalled Dictionary- Lookup" every 10 seconds on average. Then we have ca. 500 marshalled Lookups per second on the serverside (on average).
As said, a modern single CPU reaches "Full-Load" with a simple marshalled request (SendMessage-Calls going on under the hood) on ca. 4000 Calls per second. If you have an 8way-server, then you cannot extrapolate this value lineary, but you can probably at least double this value (let's say to 10000 marshalled Calls per second with an average CPU-Load over all 8 CPUs of ca 25-35%).
Now for the scenario above: 5000 concurrent users, each provoking a marshaled Dictionary-Lookup every 10 seconds, the caused CPU-Load would be: for ca. 500 marshaled lookups/sec = 1.25-1.75% if 10000 marshaled lookups/sec (worstcase) = 25-35%
So I would write a small test-scenario and check first, how it performs in your environment with marshalled calls.
Olaf
Tony Proctor - 13 Sep 2007 17:29 GMT I thought about this a little more Olaf. You cannot guarantee the performance of these objects since it depends to some extent on how their memory is spread out, and how you access it. Performance isn't bad using only hashing (up to a limit) but indexing is very bad.
Also, the type of singleton I would have to use would have to be process-specific, not machine-wide, and not specific only to the class being instantiated. With a normal VB project you'd do this via static data in a Module, but there is no shared data in a multithreaded VB context.
Tony Proctor
> > Performance of the server code is critical, and any > > cross-thread marshalling would kill performance. [quoted text clipped - 27 lines] > > Olaf Schmidt - 14 Sep 2007 14:07 GMT > I thought about this a little more Olaf. > You cannot guarantee the performance of these objects > since it depends to some extent on how their > memory is spread out, and how you access it. What do you mean with "these Objects". If you pin a singleton somewhere, then this singleton- class implements only one (private) Collection internally. And the marshaled performance I've measured is of course only related to access-attempts to the *interface* of this singleton-object. The additional time needed for accessing the internal dictionary-based Cache-Object would have to be respected/added of course - but these "extra-costs" would you see even in an unmarshaled approach, wich uses CriticalSections instead of the builtin "serialized access to a shared resource", wich OLE-Marshalling offers "for free".
A different story would be, if your shared resource is more static regarding its content and should allow parallel reading from multiple STAs - but that's what Databases are built for. If you want to have fast parallel reads from multiple STAs, and secure, serialized writes you can use e.g. SQLite, wich has low memory-footprint and offers optimized search-strategies on indexed Data and even full-text-table-searches.
In my thinking about a fast, singleton-internal Dictionary-Object I would lock even the Read-Direction for security-reasons, because a parallel Adding, whilst another STA reads Data could cause inpredictable results.
You would have to decide/measure, if a fast, but serialized Dictionary-Access performs better, than access against a DataBase-Connection, wich could process Read-Requests in parallel from multiple STAs, but has somewhat more overhead compared with a "Singleton-Dictionary".
> Performance isn't bad using > only hashing (up to a limit) but indexing is very bad. You could use e.g. my much (regarding indexed access) faster Sorted-Dictionary implementation as the private Object inside your Singleton-Class.
> Also, the type of singleton I would have to use would > have to be process-specific, not machine-wide, and > not specific only to the class being instantiated. You mean, the shared singleton-object would have to be hosted inside the IIS-Process. IIRC you can "pin" such an singleton-object using ASP in the global Application-Cache: Application.Lock Set Application("YourKey") = CreateObject("My.SingletonCache") Application.UnLock
or better yet in Application_OnStart of the Global.asa Cleanup/Shutdown then in: Application_OnEnd
retrieving the marshaled Singleton: Set MySingletonCache = Application("YourKey")
Since "transactions" only work on Call-Basis with Ole-Marshalling you should take that into respect and avoid multiple interface-calls. E.g. something like this should be avoided: If Not MySingletonCache.EntryExists(Key) Then MySingletonCache.AddEntry Key, Entry End If Or even worse: For Each-Enumerations... All those multiple calls have to be encapsulated internally.
Instead you will have to handle it in one single call: MySingletonCache.AddEntryIfNotExists Key, Entry Only this way you can be sure, that the Marshaller blocks other STAs - meaning: you have Transaction-safety - but only at "Single-Call-Level".
> With a normal VB project you'd do this via static data in a > Module, but there is no shared data in a multithreaded VB > context. You can share memory relative easy between threaded STAs in VB using SafeArray-Pointers. Of course you have to regulate the access to such shared memory-Resources with your own Locking-mechanisms using CriticalSections then. If you want, I can write an example for you, wich shares a SortedDictionary-Instance between multiple STAs in a standard VB-Exe, but regarding your IIS-host I'd recommend the *marshaled* Singleton-approach (or alternatively a lightweight DB-Engine) anyway.
Olaf
Tony Proctor - 15 Sep 2007 13:24 GMT Thanks for all this Olaf.
You're right about the Application object. I hadn't even considered that because all the Microsoft documentation is adamant that it should not be used for storing STA objects because of cross-thread issues, and the supposed serious impact on the performance of the web server.
In my most important case, the dictionary is a global read-only cache. Obviously it would have to be populated by one of the threads during startup, but after that it would only be supporting multiple readers. Hence, it would still need read-locks and write-locks to be safe but the write-lock contention would be almost non-existent.
Ideally, I wanted the dictionary items in a common memory store (to avoid the VM size getting multiplied per thread), but still allow multiple concurrent readers for maximum performance
I'll investigate your suggestions as soon as I can
Tony Proctor
> > I thought about this a little more Olaf. > > You cannot guarantee the performance of these objects [quoted text clipped - 88 lines] > > Olaf Schmidt - 15 Sep 2007 17:28 GMT > You're right about the Application object. I hadn't even > considered that because all the Microsoft documentation > is adamant that it should not be used for storing STA objects > because of cross-thread issues, and the supposed serious > impact on the performance of the web server. Yep, but that's more with regards to "inflationary usage every now and then" - but to pin a singleton there, the Application- object seems to be applicable.
> In my most important case, the dictionary is a global > read-only cache. Ah, Ok.
> Obviously it would have to be populated by one of the threads > during startup, As said, the population can very well sit inside the Singletons- Class_Initialize - and this Singleton-Class itself could be instantiated and "pinned" into the Application-object inside the Global.asa - Application_OnStart.
> but after that it would only be supporting multiple readers. > Hence, it would still need read-locks and write-locks to be > safe but the write-lock contention would be almost non-existent. If the Singleton-Content is really static (not changed during the Application-Livetime), you don't even need the read-locks.
The Singleton-Class, wich is keeped alive inside the Application-Object would only have to offer a fast way for retrieving the appropriate Array-Pointers. E.g. you could implement that, if this Singleton creates a hidden Form inside its Class_Initialize with a predefined and unique Caption. After populating two (private member-) arrays inside your singleton (one presorted String-Array for the String-Keys and one "index-matching" Array, based on a defined Private Type for the Content-Members), you could create this invisible Helper-Form and simply set two Properties on it (SetProp-API), wich contain the startpointers for the two Arrays.
Your parallel Readers inside your STAs could then retrieve these two pointers very fast using FindWindow-GetProp- APIs, and set these Pointers into safeArray-Structures inside a lightweight-Dictionary-Implementation-Class as the CacheReader in each STA - all without any Marshalling, allowing multiple and *parallel* Read-Access against the so shared Arrays, hosted in the Singleton-Class wich is kept alive in the Application-Object. These lightweight Access-Classes would then be able, to use the vitually spanned safeArrays for very fast indexed access - and due to the presorted nature of the Keys in the String-Array you could also achieve very fast keyed-access using a simple Binary-Chop against the virtually spanned KeyString-Array.
Olaf
Tony Proctor - 15 Sep 2007 19:26 GMT I agree with most of what you said Olaf, except the 'hidden Form'. I don't believe that's really necessary, and a project with 'Retain in Memory' & 'Unattended execution' cannot have any UI elements in it anyway.
Tony Proctor
> > You're right about the Application object. I hadn't even > > considered that because all the Microsoft documentation [quoted text clipped - 51 lines] > > Olaf Schmidt - 15 Sep 2007 19:59 GMT > I agree with most of what you said Olaf, except the > 'hidden Form'. I don't believe that's really necessary, > and a project with 'Retain in Memory' & 'Unattended > execution' cannot have any UI elements in it anyway. Thought about the CreateWindowEx-API instead of "VBRuntime-Controlled Forms". This should work - even in 'Unattended execution-mode' - the whole OLE- Marshalling is using hidden forms under the hood - and this works well, even without any loaded desktop in "pure service-mode".
But you are right, there are of course additional ways, to make two pointer-values (and maybe the Array-Count too) available in a fast way (without any marshalled calls) to STAs, wich need them.
Regards,
Olaf
Tony Proctor - 18 Oct 2007 21:15 GMT This all breaks down Olaf because you cannot store STA objects in the ASP Application object
Tony Proctor
> > You're right about the Application object. I hadn't even > > considered that because all the Microsoft documentation [quoted text clipped - 51 lines] > > Olaf Schmidt - 19 Oct 2007 01:46 GMT > This all breaks down Olaf because you cannot store STA > objects in the ASP Application object Ah Ok - long time ago, since I used the IIS and its environment (in some older version this has worked).
Ok, then try this approach: (just tested this with an IIS, wich was installed and updated with VS2005-Express - and it works well, no global.asa, no special initializing)
Simply use an *.asp-script (called from a browser) for testing. <% Server.CreateObject("Your.ProgID").DoIt Response %>
You can adjust the UserEntries in the Demo and watch the Memory-consumption of dllhost.exe in the TaskManager.
'And this goes into a Public Class in an AX-Dll Option Explicit
Private Type TGlob 'always use fixed length strings inside the Types Initiated As String * 10 LastThreadID As Long LastAccess As Date MaxArrEntryCount As Long CurArrEntryCount As Long '... End Type
Private Type TGlobArray 'always use fixed length strings inside the Types ID As Long UserName As String * 128 '... End Type
Private Declare Sub RtlMoveMemory Lib "kernel32" _ (Dst As Any, Src As Any, ByVal bc&) Private Declare Function ArrPtr& Lib "msvbvm60" Alias "VarPtr" _ (Arr() As Any) Private Declare Function CloseHandle& Lib "kernel32" _ (ByVal hObj&) Private Declare Function CreateFileMappingA& Lib "kernel32" _ (ByVal hFile&, Attr As Any, ByVal Protect&, ByVal SizeHigh&, _ ByVal SizeLow&, ByVal MapName$) Private Declare Function MapViewOfFile& Lib "kernel32" _ (ByVal hFile&, ByVal DesAccess&, ByVal OffsHigh&, ByVal OffsLow&, _ ByVal NumberOfBytesToMap&) Private Declare Function UnmapViewOfFile& Lib "kernel32" _ (ByVal lpBaseAddress As Long)
Private Const MapKey$ = "MyMapKey", FILE_MAP_ALL_ACCESS& = &HF001F
Private Const MaxUsers& = 100
Private Glob() As TGlob, saGlob&(5) Private Users() As TGlobArray, saUsers&(5) Private BaseAddr&, hFile&, Initiator As Boolean
Public Sub DoIt(Response) Dim i As Long, NewUserIdx As Long
If BaseAddr = 0 Then Exit Sub
With Glob(0) Response.Write "<PRE>" Response.Write "Global BaseData:<BR>" Response.Write " LastThreadID: " & .LastThreadID & "<BR>" Response.Write " CurrentThreadID: " & App.ThreadID & "<BR>" Response.Write " LastAccess: " & .LastAccess & "<BR>" Response.Write " CurrentAccess: " & Now & "<BR><BR>"
.LastThreadID = App.ThreadID .LastAccess = Now
Response.Write "We have enough memory for: " & _ .MaxArrEntryCount & " Users<BR><BR>" 'lets Add another User NewUserIdx = .CurArrEntryCount If NewUserIdx < .MaxArrEntryCount Then .CurArrEntryCount = NewUserIdx + 1 'Now we have time, to fill in the members at the NewUserIdx With Users(NewUserIdx) .ID = NewUserIdx + 1 .UserName = "UserName_" & .ID & vbNullChar End With NewUserIdx = NewUserIdx + 1 End If
Response.Write "Current UserList has: " & NewUserIdx & " Users<BR>" For i = 0 To NewUserIdx - 1 With Users(i) Response.Write " UserID: " & .ID & _ ", UserName: " & F2S(.UserName) & "<BR>" End With Next i Response.Write "</PRE>" End With End Sub
Private Sub Class_Initialize() Dim GlobDummy As TGlob, MapBaseSize& Dim GlobArrayDummy As TGlobArray, MapUserSize& MapBaseSize = LenB(GlobDummy) 'always use LenB MapUserSize = LenB(GlobArrayDummy) * MaxUsers 'always use LenB
'Initialize safearray-descriptor saGlob(0) = 1 'Dimension saGlob(1) = LenB(GlobDummy) 'size of Member saGlob(4) = 1 'Number of Array-Elements 'array-binding (bind descriptor to the Glob-Arr) RtlMoveMemory ByVal ArrPtr(Glob), VarPtr(saGlob(0)), 4
'now the same for the User-Array saUsers(0) = 1 'Dimension saUsers(1) = LenB(GlobArrayDummy) 'size of Member saUsers(4) = MaxUsers 'Number of Array-Elements 'array-binding (bind descriptor to the Users-Arr) RtlMoveMemory ByVal ArrPtr(Users), VarPtr(saUsers(0)), 4
'do the Mapping inside the SwapFile hFile = CreateFileMappingA(-1, ByVal 0&, 4, 0&, _ MapBaseSize + MapUserSize, MapKey) If hFile = 0 Then Exit Sub
BaseAddr = MapViewOfFile(hFile, FILE_MAP_ALL_ACCESS&, 0&, 0&, 0&) If BaseAddr = 0 Then Exit Sub
saGlob(3) = BaseAddr 'set the DataPointer to the mapped area If F2S(Glob(0).Initiated) <> "Initiated" Then Initiator = True Glob(0).Initiated = "Initiated" & vbNullChar Glob(0).MaxArrEntryCount = MaxUsers End If
'finally we set the Users-DataPointer behind the Glob-Data saUsers(3) = BaseAddr + MapBaseSize End Sub
Private Sub Class_Terminate() RtlMoveMemory ByVal ArrPtr(Glob), 0&, 4 'reset array-binding RtlMoveMemory ByVal ArrPtr(Users), 0&, 4 'reset array-binding
If Not Initiator Then 'if we are not the initiator, let's cleanup here If BaseAddr Then UnmapViewOfFile BaseAddr: BaseAddr = 0 If hFile Then CloseHandle hFile: hFile = 0 End If End Sub
Private Function F2S$(FLStr$) 'Helper to convert fixed to BSTR Dim Pos& Pos = InStr(FLStr, vbNullChar) If Pos Then F2S = Left$(FLStr, Pos - 1) End Function
Olaf
Tony Proctor - 19 Oct 2007 17:25 GMT My Dictionaries (there are several) could have 20k-30k items so a binary chop wouldn't be quick enough Olaf.
I've tried using an Application Singleton created via GLOBAL.ASA but having other issues there (see more recent thread in this group). If that fails, is there a way of using the ROT to hold a reference to an object within just a specific process? What I want to avoid is the risk of multiple web applications that use the same DLL accidentally sharing the same instance of my singleton. In other words, can the ROT be used to implement a "process-wide singleton" rather than a "machine-wide singleton"?
Tony Proctor
> > This all breaks down Olaf because you cannot store STA > > objects in the ASP Application object [quoted text clipped - 150 lines] > > Olaf Schmidt - 19 Oct 2007 19:12 GMT > My Dictionaries (there are several) could have 20k-30k items > so a binary chop wouldn't be quick enough Olaf. For 20-30k items the bin-search would only need to do ca. 12 string-compares on average to find a record over the appropriate String-Key.
> I've tried using an Application Singleton created via GLOBAL.ASA > but having other issues there (see more recent thread in this group). Yea, seen that - found especially interesting, that the "outside WebRoot" Dlls caused less problems than the "insiders" - one would expect that exactly the other way round.
> If that fails, is there a way of using the ROT to hold a reference > to an object within just a specific process? Not really, but if you google for one of my ROT-postings here in this group (ca. two years ago), then you will find a routine, wich is able to place and retrieve a VB-Class in/from the ROT over a free definable FileMoniker(String) - this way you could achieve different instances of the same Cache-Class for different processes (wich simply would have to know "their key").
> In other words, can the ROT be used to implement a > "process-wide singleton" rather than a "machine-wide singleton"? As said, not directly, but over a special "ROT-Key". The problem with the ROT-Objects is, that you will have to see, if the IIS allows their instantiation/registration in the ROT and that there are a few problems with them regarding "pure service mode" - meaning if the OS is not bootet into a Window-Station or -Desktop. But the code I've mentioned above does to try to take that into respect IIRC.
With my just posted example here, there is only coding to do - no surprises possible (if coded correctly). The singleton-approach (be it in the ROT or the global.asa) could offer some additional fun, even if you've managed the "singleton-instantiation-problems".
Its something like a "pronounced traffic jam" where you know about a definitely "jam-free", alternative way, wich on the other hand would be some miles longer to drive. ;-)
Olaf
Tony Proctor - 19 Oct 2007 19:30 GMT This sounds promising. I've used the ROT in other contexts Olaf, but I always use the classid as the key - I didn't know if was freely definable. Do you have a reference for that post?
I didn't understand the reference about booting into a window station. The other contexts I've used the ROT are purely server-side (no UI), just like this new ASP requirement.
Tony Proctor
> > My Dictionaries (there are several) could have 20k-30k items > > so a binary chop wouldn't be quick enough Olaf. [quoted text clipped - 39 lines] > > Olaf Schmidt - 19 Oct 2007 20:53 GMT > This sounds promising. I've used the ROT in other > contexts Olaf, but I always use the classid as the key - > I didn't know if was freely definable. > Do you have a reference for that post? Just search inside google.groups with these keywords: rot vb sss matthew
> I didn't understand the reference about booting into a > window station. Just follow the thread (you were taking part BTW)... ;-)
Olaf
Tony Proctor - 20 Oct 2007 10:50 GMT Ah, that thread! Yes, at the moment I only use RegisterActiveObject for working with the ROT since I don't need any extra typelibs. However, the downside is that it doesn't offer file-moniker binding.
Since a file-moniker must appear in the registry then I have to consider the possibility of accidentally leaving artefacts in there since an ASP app doesn't really have a "close down" (Session-end and Application-end are useless for anything like that). This means I cannot use a transient differentiator like the process ID. I probably need something related to the relevant virtual directory, which in turn is configured by customers, not myself.
Tony Proctor
> > This sounds promising. I've used the ROT in other > > contexts Olaf, but I always use the classid as the key - [quoted text clipped - 8 lines] > > Olaf Tony Proctor - 10 Sep 2007 19:05 GMT ...and, I forgot to say: I was hoping to find one off-the-shelf :-)
Tony Proctor
> > They would be 'simple data types' Olaf, i.e mainly text and > > numbers. Hence, no Objects, and so no apartment violations :-) [quoted text clipped - 23 lines] > > Olaf
|
|
|