WineHQ
Bug Tracking Database – Bug 31406

 Bugzilla

 

Last modified: 2013-01-22 07:17:46 CST  

Borderlands freezes in game when items details are displayed

Bug 31406 - Borderlands freezes in game when items details are displayed
Borderlands freezes in game when items details are displayed
Status: CLOSED NOTOURBUG
AppDB: Show Apps affected by this bug
Product: Wine
Classification: Unclassified
Component: directx-d3d
1.5.10
x86 Linux
: P2 normal
: ---
Assigned To: Mr. Bugs
: regression
: 31578 31602 31633 31849 31883 31901 31943 32014 32607 (view as bug list)
Depends on:
Blocks: 31754 31589
  Show dependency tree
 
Reported: 2012-08-04 20:02 CDT by John P Sims
Modified: 2013-01-22 07:17 CST (History)
22 users (show)

See Also:
Regression SHA1: 8a95eb060449e93bc38af1cabc928f0a959e9e14
Fixed by SHA1:
Distribution: ---
Staged patchset:


Attachments
test case to trigger the hanging (opengl) (72.54 KB, application/x-7z-compressed)
2012-08-19 04:45 CDT, Rico
Details
Add the X11/GL lock back around XCheckIfEvent (5.65 KB, patch)
2012-09-08 23:01 CDT, Erich E. Hoover
Details | Diff
Remove locking around libxcb poll() (855 bytes, patch)
2012-09-09 19:33 CDT, Erich E. Hoover
Details | Diff
Use separate locking for libxcb poll() (1.40 KB, patch)
2012-09-10 16:22 CDT, Erich E. Hoover
Details | Diff
Fix for the libxcb poll() issue (1.10 KB, patch)
2012-09-12 09:47 CDT, Erich E. Hoover
Details | Diff
Backtrace crash but no hang reported (2.55 KB, application/x-lzma)
2012-09-17 03:10 CDT, Chadwick Ferguson
Details
Backtrace crash but no hang reported (2.54 KB, application/x-lzma)
2012-09-17 03:11 CDT, Chadwick Ferguson
Details
Fix for the libxcb poll() issue - debug version (1.33 KB, patch)
2012-09-17 04:27 CDT, Rico
Details | Diff
Borderlands, steam, freez (645.24 KB, text/plain)
2012-09-18 01:47 CDT, MonoBOY
Details

Note You need to log in before you can comment on or make changes to this bug.
Description John P Sims 2012-08-04 20:02:59 CDT
Borderlands now locks up when ever you look at the details of an in game item (gun, shield, etc) in the game world or in the inventory menu. It is a regression. Here are the results of the regression test:


8a95eb060449e93bc38af1cabc928f0a959e9e14 is the first bad commit
commit 8a95eb060449e93bc38af1cabc928f0a959e9e14
Author: Henri Verbeet <hverbeet@codeweavers.com>
Date:   Wed Jul 18 21:32:33 2012 +0200

    wined3d: Disable the X11 / GL lock.

:040000 040000 ffc792739741b2aa3999dc533803e600ace0b3fd c37d9d0fe4165e09a89c9e22066e3acc17537bbb M	dlls

Reverting this patch eliminates the lock up and allows the game to continue running as it had previously.
Comment 1 Benn Snyder 2012-08-14 18:19:50 CDT
I can confirm this.  After the upgrade to wine-1.5.10, Borderlands began freezing when checking item details, during the intro, and when I minimized the window.

I applied the inverse of
wined3d: Disable the X11 / GL lock 
http://www.winehq.org/pipermail/wine-patches/2012-July/116361.html
after which Borderlands is running fine again.


nVidia Corporation G98M [GeForce G 105M] (rev a1)
x11-drivers/nvidia-drivers-295.59

app-emulation/wine-1.5.10 was built with the following:
USE="X alsa gecko jpeg lcms ldap mono mp3 (multilib) ncurses nls opengl oss perl png pulseaudio ssl threads truetype v4l win32 win64 xcomposite xml -capi -cups -custom-cflags -fontconfig -gnutls -gphoto2 -gsm (-gstreamer) -hardened -odbc -openal -opencl -osmesa -samba -scanner (-selinux) -test -udisks -xinerama"
CFLAGS="-march=native -O2 -pipe -O2"
CXXFLAGS="-march=native -O2 -pipe -O2"
Comment 2 Henri Verbeet 2012-08-15 01:18:26 CDT
Yeah, I think I can reproduce this, although it's not actually that consistent at which point in the game it ends up hanging. It seems to hang inside the driver, in a call to glBindFramebuffer() with a newly created FBO, on xcb_wait_for_reply(). What makes this a bit hard to debug is that e.g. a FIXME or TRACE right before that glBindFramebuffer() appears to be enough of a barrier to prevent the hang from happening, making it hang somewhere else.
Comment 3 DL 2012-08-19 01:56:47 CDT
Spec Ops: The Line also crashes consistently when attempting to load the main menu. Running a regression test and I found the same commit:

8a95eb060449e93bc38af1cabc928f0a959e9e14 is the first bad commit
commit 8a95eb060449e93bc38af1cabc928f0a959e9e14
Author: Henri Verbeet <hverbeet@codeweavers.com>
Date:   Wed Jul 18 21:32:33 2012 +0200

    wined3d: Disable the X11 / GL lock.

:040000 040000 ffc792739741b2aa3999dc533803e600ace0b3fd c37d9d0fe4165e09a89c9e22066e3acc17537bbb M      dlls
Comment 4 Rico 2012-08-19 04:45:52 CDT
Created attachment 41417 [details]
test case to trigger the hanging (opengl)

I have the same problem when using the attached test case.
1. wine graphics.exe
2. hit run button
3. move the propsheetdlg over the "3D"-window and a hang will happen

The test case uses opengl which was broken since commit 312dd5297b7edfaab059ac2a75d61a5d1ec73664 (but commit 8a95eb060449e93bc38af1cabc928f0a959e9e14 broke the same test for D3D8/9 - off course using d3d calls instead of opengl ones). I think they are somehow related, so I added the test here. I could add a test for d3d if that's needed, but I think it just moves the problem one step away from the source (with the additional layer). Also I think the mentioned patch just revealed the already existing problem. Also note the hang doesn't have to be specific to a special opengl function ... on several tries I got it hanging in glClear or glViewport. All called xcb_wait_for_reply.

Using Fedora 14 with a radeon R600:
OpenGL vendor string: Advanced Micro Devices, Inc.
OpenGL renderer string: Mesa DRI R600 (RV710 954F) 20090101  TCL DRI2
OpenGL version string: 2.1 Mesa 7.9
OpenGL shading language version string: 1.20
Comment 5 Henri Verbeet 2012-08-19 07:15:00 CDT
(In reply to comment #4)
> Using Fedora 14 with a radeon R600:
> OpenGL vendor string: Advanced Micro Devices, Inc.
> OpenGL renderer string: Mesa DRI R600 (RV710 954F) 20090101  TCL DRI2
You want to be using fairly current versions of r600g, but that's probably not the issue here.

My guess would be that this an issue with libX11 over XCB and not necessarily specific to a particular driver (but still technically UPSTREAM). It may be interesting to see if it works any better with a libX11 built with --without-xcb, although I could also very well imagine all kinds of things depending on being able to mix XLib and XCB at this point.

You may also want to try to simplify that test case a bit more and port it to pure GLX / XLib.
Comment 6 rkfg 2012-08-29 07:17:39 CDT
Reverting the regression doesn't work anymore on 1.5.11. I get this:

../../tools/winegcc/winegcc -m32 -B../../tools/winebuild --sysroot=../.. -fasynchronous-unwind-tables -shared ./wined3d.spec arb_program_shader.o ati_fragment_shader.o buffer.o context.o device.o directx.o drawprim.o gl_compat.o glsl_shader.o nvidia_texture_shader.o palette.o query.o resource.o shader.o shader_sm1.o shader_sm4.o state.o stateblock.o surface.o swapchain.o texture.o utils.o vertexdeclaration.o view.o volume.o wined3d_main.o      version.res    -o wined3d.dll.so  -luuid -luser32 -lgdi32 -ladvapi32  ../../libs/port/libwine_port.a   
surface.o: In function `ffp_blit_unset':
/tmp/wine-git/dlls/wined3d/surface.c:6406: undefined reference to `wine_tsx11_lock_ptr'
/usr/bin/ld: surface.o: relocation R_386_GOTOFF against undefined hidden symbol `wine_tsx11_lock_ptr' can not be used when making a shared object
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
winegcc: gcc failed

Seems those functions were removed completely or I'm doing something wrong.
Comment 7 Rico 2012-08-30 08:01:10 CDT
Sadly, the option --without-xcb for libX11 was disabled two years ago (see http://cgit.freedesktop.org/xorg/lib/libX11/commit/configure.ac?id=15e5eaf62897b3179d1fbe457cb19f886f0449f8 ).
Comment 8 Henri Verbeet 2012-08-30 08:16:41 CDT
Yeah, but you can probably still build an old version like 1.3.6.
Comment 9 Rico 2012-08-31 13:22:10 CDT
As workaround, please try: WINEDEBUG=+synchronous
Comment 10 John P Sims 2012-08-31 23:08:31 CDT
Rico,

My apologies if I'm speaking out of turn, but WINEDEBUG=+synchronous doesn't appear to be a workaround. In wine-1.5.11, the in game freeze still occurs. I'll try 1.5.12 and report back.
Comment 11 Alexandre Julliard 2012-09-02 13:09:45 CDT
*** Bug 31602 has been marked as a duplicate of this bug. ***
Comment 12 John P Sims 2012-09-05 22:17:06 CDT
Using wine-1.5.12, WINEDEBUG=+synchronous doesn't appear to work around the freezing issue in Borderlands.
Comment 13 Rico 2012-09-06 02:04:05 CDT
John, could you try my test case and see if it hangs on your system. If so could you try if WINEDEBUG=+synchronous fixes it for you? It may be another bug, but it also may be the same and behave differently depending on some libs.

There are other possible workarounds, but they both are definitively not the way to do it for a game. I'll just list them, but take it with care.
1. Build libX11 without libxcb.
2. For me with mesa LIBGL_ALWAYS_INDIRECT seems to do it, too.
Comment 14 Rico 2012-09-06 05:25:36 CDT
I think this is somehow related:
https://bugs.freedesktop.org/show_bug.cgi?id=20708 ... three and a half year old...
Comment 15 John P Sims 2012-09-06 23:28:32 CDT
Rico, graphics.exe does not hang the system. I get a green window. Using WINEDEBUG=+synchronous produces the same result.

Compiling a new libX11 seems a little drastic. I mean, I am comfortable building everything in the Xorg tarball, don't get me wrong, but the last one I built was like Xorg 7.5. It's my understanding that Xorg 7.7 has changed things quite a bit since then, but I digress.

So, help me here, what exactly would building a custom libX11 prove? Does this help identify a problem with wine or Xorg? Will building a libX11 without libxcb help fix wine?

If building libX11 without xcb will help wine devs, I'll do it, but I don't need to play borderlands *that* badly.

As to the last tip, I don't use mesa. I'm one of those dirty NVIDIA proprietary blob users. ;-)
Comment 16 Rico 2012-09-07 03:22:20 CDT
Well, yeah compiling libX11 without libxcb support isn't easy these days. But I think the gateway (X11-xcb) is broken. It would prove, if your problem and my problem have the same source, nothing more, which till now is just a guess. As it seems the problem is not in wine, it is in libX11 and thus technically UPSTREAM. We take this bug to cover information about the problem and to catch dupes.

I think building without xcb won't help anyone, besides you cold try the game and it may work. As of #14 there it looks like are some real problems with the way the gateway works.

I think there are 2 solutions for us:
1. fix libX11-xcb gateway (doesn't seem to be easy)
2. use xcb directly (not sure if that is better, but we may give it a try)

For users:
1. use wine-1.5.9
2. use wine git & revert the problematic changes
3. use WINEDEBUG=+synchronous or LIBGL_ALWAYS_INDIRECT (both may or may not work)
Comment 17 Alexandre Julliard 2012-09-07 04:11:12 CDT
*** Bug 31633 has been marked as a duplicate of this bug. ***
Comment 18 Erich E. Hoover 2012-09-07 13:21:56 CDT
(In reply to comment #14)
> I think this is somehow related:
> https://bugs.freedesktop.org/show_bug.cgi?id=20708 ... three and a half year
> old...

It looks like that bug got wrapped into https://bugs.freedesktop.org/show_bug.cgi?id=30450 - did you try the patch in there?
Comment 19 Erich E. Hoover 2012-09-08 10:37:51 CDT
Ok, we confirmed yesterday (conversations on IRC) that this is not either of the X bugs previously listed, so I've opened up a new freedesktop.org bug with a request for help (https://bugs.freedesktop.org/show_bug.cgi?id=54671).  The problem appears to occur when both OpenGL and X11 are in use simultaneously and they happen to use the same sequence number for two separate threads (based on watching the Netflix Silverlight app).
Comment 20 Erich E. Hoover 2012-09-08 17:01:24 CDT
(In reply to comment #4)
> Created attachment 41417 [details]
> test case to trigger the hanging (opengl)
> 
> I have the same problem when using the attached test case.
> 1. wine graphics.exe
> 2. hit run button
> 3. move the propsheetdlg over the "3D"-window and a hang will happen
> ...
> OpenGL renderer string: Mesa DRI R600 (RV710 954F) 20090101  TCL DRI2
> ...

I should probably note that I don't get a lockup with your test case, but I'm using the nVidia proprietary driver.
Comment 21 Erich E. Hoover 2012-09-08 23:01:41 CDT
Created attachment 41611 [details]
Add the X11/GL lock back around XCheckIfEvent

Ok, I've narrowed down which routine is responsible for the lockup (XCheckIfEvent).  The attached patch is a hack to add the lock back around this routine (against latest git).

I've also noticed that (at least for Borderlands, though not for Netflix) using libxcb-1.8.1 also fixes the issue.  You can download and install libxcb-1.8.1 separate from the rest of the X11 libraries, so it should be possible for distros to push this update and have Wine put a dependency on it.
Comment 22 Dan Kegel 2012-09-08 23:06:57 CDT
According to http://packages.ubuntu.com/source/precise/libxcb
ubuntu 12.04 and up already have libxcb 1.8.1, so they should be ok
already, right?
Comment 23 Erich E. Hoover 2012-09-08 23:16:26 CDT
(In reply to comment #22)
> According to http://packages.ubuntu.com/source/precise/libxcb
> ubuntu 12.04 and up already have libxcb 1.8.1, so they should be ok
> already, right?

Probably, I ran it 5 times each way - with 1.8 or lower it hung 100% of the time (without the hack), with 1.8.1 it hung 0% of the time.  I'm moved up to xorg-edgers to try and work on this bug, unfortunately they're only up to 1.8.  It's really easy to build libxcb, so I highly recommend giving it a try.  You can install the build dependencies with the package manager, after that it's just using "-m32" in the CFLAGS when configuring and adding /usr/local/lib to your LD_LIBRARY_FLAGS so that Wine will pick up on the new library.
Comment 24 Erich E. Hoover 2012-09-08 23:52:38 CDT
(In reply to comment #23)
> Probably, I ran it 5 times each way - with 1.8 or lower it hung 100% of the
> ...

Actually, double checking it looks like I compiled 1.8 without "-m32" (it should work as well).  xorg-edgers must not include a 32-bit version of 1.8...
Comment 25 Erich E. Hoover 2012-09-09 00:31:59 CDT
(In reply to comment #24)
> (In reply to comment #23)
> > Probably, I ran it 5 times each way - with 1.8 or lower it hung 100% of the
> > ...
> 
> Actually, double checking it looks like I compiled 1.8 without "-m32" (it
> should work as well).  xorg-edgers must not include a 32-bit version of 1.8...

For the curious, it appears to be fixed by xcb commit 5ceeaaa4294201b3f613c07f9ec610c0e5f673c7 (Fix a dead-lock due to xcb_poll_for_reply).  If you're running a newer version of xcb then it shouldn't be a problem for Borderlands (though I'm currently back on libx11 2:1.4.4-2ubuntu1, so it's possible there's a different interaction).
Comment 26 Rico 2012-09-09 02:57:36 CDT
Well mesa on R600 uses libx11 which uses libxcb, that's why it may hang for me and my test case. Probably nvidia uses it in another way.

The attached patch doesn't fix my issue (so it might not have the same source).

Commit 5ceeaaa4294201b3f613c07f9ec610c0e5f673c7 is prior libxcb 1.8. John / Benn / DL what's your libxcb and libX11 version?
Comment 27 Rico 2012-09-09 03:19:28 CDT
Hence, the patch is not able to solve my test problem, because I'm using OpenGL directly. But off course it fixes it when I use d3d8/9.
Comment 28 Erich E. Hoover 2012-09-09 10:07:37 CDT
(In reply to comment #26)
> ...
> Commit 5ceeaaa4294201b3f613c07f9ec610c0e5f673c7 is prior libxcb 1.8. John /
> Benn / DL what's your libxcb and libX11 version?

Try compiling libxcb yourself, I have a sneaking suspicion that the Ubuntu repo isn't including a 32-bit libxcb 1.8 if xorg-edgers isn't.
Comment 29 Benn Snyder 2012-09-09 13:45:58 CDT
(In reply to comment #26)
> Commit 5ceeaaa4294201b3f613c07f9ec610c0e5f673c7 is prior libxcb 1.8. John /
> Benn / DL what's your libxcb and libX11 version?

I have x11-libs/libxcb-1.8.1, x11-libs/libX11-1.5.0, and x11-drivers/nvidia-drivers-295.71.  I'd like to test this again, but now with wine-1.5.12 I can't even get Steam to start.  Downgrading to 1.5.11 now.
Comment 30 Erich E. Hoover 2012-09-09 14:03:27 CDT
(In reply to comment #29)
> (In reply to comment #26)
> > Commit 5ceeaaa4294201b3f613c07f9ec610c0e5f673c7 is prior libxcb 1.8. John /
> > Benn / DL what's your libxcb and libX11 version?
> 
> I have x11-libs/libxcb-1.8.1, x11-libs/libX11-1.5.0, and
> x11-drivers/nvidia-drivers-295.71.  I'd like to test this again, but now with
> wine-1.5.12 I can't even get Steam to start.  Downgrading to 1.5.11 now.

I've had to disable the "dwrite" library to run Steam on recent Wine versions, though I haven't checked that that's still the case.
Comment 31 Erich E. Hoover 2012-09-09 19:33:55 CDT
Created attachment 41627 [details]
Remove locking around libxcb poll()

Ok, if I'm right the issue is that the poll() in xcb is getting stuck because the fd is being serviced by another thread.  The attached patch (hack?) pulls out the code that disables the IO lock around poll()ing.
Comment 32 Rico 2012-09-10 02:40:29 CDT
The patch "Remove locking around libxcb poll()" doesn't work for my case.
Comment 33 Chiitoo 2012-09-10 09:20:30 CDT
Teegrins~

I was creating a new entry, but as I was getting finished, I noticed the same commit I ended up to with a bisect is mentioned here, so I'll rather add my findings here as well.

Occasionally launching the PlayOnline Viewer (polboot.exe/pol.exe/FINAL FANTASY XI) will result into a black box, with the background music playing.  No graphics are rendered, and no sound is produced by moving the mouse around where the options would be (or using the game-pad to go through different options), so no interaction with the application is possible, except for being able to minimize/maximize the application by clicking the Task Manager entry, or from the window icons.

This would sometimes happen when logging out of FINAL FANTASY XI to return to the PlayOnline Viewer as well.

Sometimes there might not be music even, and the application doesn't respond in any way, not even when clicking the Task Manager entry.  In any case, there doesn't seem to be any interesting terminal-output, at least not without d3d channel enabled.

Due to the more or less random nature of the issue, I can not be 100% certain that the following is indeed the first bad commit, but I did bisect it two times and I would end up with the same results each time:


8a95eb060449e93bc38af1cabc928f0a959e9e14 is the first bad commit
commit 8a95eb060449e93bc38af1cabc928f0a959e9e14
Author: Henri Verbeet <hverbeet@codeweavers.com>
Date:   Wed Jul 18 21:32:33 2012 +0200

    wined3d: Disable the X11 / GL lock.

:040000 040000 ffc792739741b2aa3999dc533803e600ace0b3fd c37d9d0fe4165e09a89c9e22066e3acc17537bbb M      dlls


I wonder if it might be easier to debug with due to the fact that it happens right when launching the application, instead of in-game as it seems to happen for Borderlands.  That is, if the real issue is the same in the first place.

Let me know if I should still open a dedicated bug for this, and/or if I should add a back trace or more, though it seems like it might be redundant at this point.

I'll at least add a link to it as it seems related?
Comment 34 Erich E. Hoover 2012-09-10 11:03:22 CDT
(In reply to comment #33)
>...
> Occasionally launching the PlayOnline Viewer (polboot.exe/pol.exe/FINAL FANTASY
> XI) will result into a black box, with the background music playing.  No
> graphics are rendered, and no sound is produced by moving the mouse around
> where the options would be (or using the game-pad to go through different
> options), so no interaction with the application is possible, except for being
> able to minimize/maximize the application by clicking the Task Manager entry,
> or from the window icons.
> ...

What version of libxcb do you have?  Do you think you could try the patch from comment #31?
Comment 35 Erich E. Hoover 2012-09-10 16:22:46 CDT
Created attachment 41639 [details]
Use separate locking for libxcb poll()

(In reply to comment #32)
> The patch "Remove locking around libxcb poll()" doesn't work for my case.

It improves my confidence in the problem that you got this patch working (convo on IRC).  Since the problem is unique to multiple simultaneous poll()s, it might be possible to get the performance back most of the way by having a separate lock just for the poll().  So, please try the attached patch and see how that turns out.
Comment 36 Chiitoo 2012-09-10 20:11:01 CDT
(In reply to comment #34)

> What version of libxcb do you have?  Do you think you could try the patch from
> comment #31?

x11-libs/libxcb-1.8.1 was built with the following:
USE="(multilib) -doc (-selinux) -static-libs"

I tried the patch from 31 as well as 35, but unfortunately, they seem to have no effect in my case.  Will do some more testing later/earlier today...
Comment 37 Erich E. Hoover 2012-09-10 20:27:27 CDT
(In reply to comment #36)
> ...
> I tried the patch from 31 as well as 35, but unfortunately, they seem to have
> no effect in my case.  Will do some more testing later/earlier today...

You probably need to use CFLAGS="-m32" to compile for 32-bit apps and use LD_LIBRARY_PATH="/usr/local/lib:/usr/lib:/lib" when launching Wine to get it to use your personally-compiled version.
Comment 38 Chiitoo 2012-09-10 20:39:52 CDT
(In reply to comment #37)
> (In reply to comment #36)
> > ...
> > I tried the patch from 31 as well as 35, but unfortunately, they seem to have
> > no effect in my case.  Will do some more testing later/earlier today...
> 
> You probably need to use CFLAGS="-m32" to compile for 32-bit apps and use
> LD_LIBRARY_PATH="/usr/local/lib:/usr/lib:/lib" when launching Wine to get it to
> use your personally-compiled version.

Sorry, if I misunderstand but, I use Gentoo, and the way I went with this was that I used the package-manager itself, Portage, to apply the patch before compiling and installing (the way everything is usually done in Gentoo).  So there was no multiple versions to be used, just the one and only.

However, as I mentioned, I do need to try it out some more later with some more time in my hands.


On another note, a version of the PlayOnline Viewer can be downloaded here: http://www.playonline.com/ff11eu/download/media/vista01.html?pageID=media

I tested it just now, with a clean Wine prefix to boot, and was indeed able to re-produce the issue.  Actually with the latest git-version, it seems like it isn't able to launch at all!  (And just as I had wrote that, I tried one more time and it did launch, typical.  Either way, that must be about 1/20+ tries.)
Comment 39 t800 2012-09-10 21:36:08 CDT
I applied these patches on my Gentoo and Mass Effect 2/3 now run again fine. 
Thanks for these patches! You're make me day!
Comment 40 t800 2012-09-10 21:47:56 CDT
For patching i used wine version is 1.5.12 and libxcb's version is 1.8.1

I used x11-drivers/nvidia-drivers-304.43 and my gfx card is NVIDIA 120 GT


Sorry for my bad English :)
Comment 41 Rico 2012-09-11 02:58:11 CDT
The patch "Use separate locking for libxcb poll()" works fine for my test case (as the other two patches - sorry for the wrong test results). The speed looks nearly the same as without the patch.

Thought, I'd probably destroy the mutex:

diff --git a/src/xcb_conn.c b/src/xcb_conn.c
index 7979491..8be1849 100644
--- a/src/xcb_conn.c
+++ b/src/xcb_conn.c
@@ -313,6 +314,7 @@ void xcb_disconnect(xcb_connection_t *c)
     close(c->fd);
 
     pthread_mutex_destroy(&c->iolock);
+    pthread_mutex_destroy(&c->pllock);
     _xcb_in_destroy(&c->in);
     _xcb_out_destroy(&c->out);
Comment 42 Alexandre Julliard 2012-09-11 03:39:53 CDT
*** Bug 31578 has been marked as a duplicate of this bug. ***
Comment 43 Erich E. Hoover 2012-09-11 08:41:59 CDT
(In reply to comment #41)
> The patch "Use separate locking for libxcb poll()" works fine for my test case
> (as the other two patches - sorry for the wrong test results). The speed looks
> nearly the same as without the patch.
> 
> Thought, I'd probably destroy the mutex:
> ...

Good call :)  I'm surprised that this actually works, I would guess that it really just changes the timing enough that things don't barf anymore.  I'll relay this on to the xcb folks and see what they tell us.
Comment 44 Erich E. Hoover 2012-09-12 09:47:40 CDT
Created attachment 41657 [details]
Fix for the libxcb poll() issue

Ok, I've had some help from the XCB folks and it looks like the problem is that a thread with a writing request can snipe the poll() from a thread with a reading request.  The attached patch fixes that issue in my case, but there is some concern that it won't work in certain circumstances.  Please let me know if this works out ok (for everyone here that is capable of building their own libxcb).
Comment 45 Chiitoo 2012-09-12 11:38:37 CDT
Hmmm hrm, does not seem to affect me (or rather PlayOnline Viewer).  Perhaps the issue in my case is something different after all, though it is peculiar that the same commit would affect it in rather similar way(s)...
Comment 46 Erich E. Hoover 2012-09-12 12:05:25 CDT
(In reply to comment #45)
> Hmmm hrm, does not seem to affect me (or rather PlayOnline Viewer).  Perhaps
> the issue in my case is something different after all, though it is peculiar
> that the same commit would affect it in rather similar way(s)...

Yeah, that seems unlikely...  Maybe t800 can help you test the patch?  I'm not very familiar with Gentoo, so I'm afraid I would not be very helpful.
Comment 47 Matteo Bruni 2012-09-13 09:56:54 CDT
(In reply to comment #44)
> Created attachment 41657 [details]
> Fix for the libxcb poll() issue
> 
> Ok, I've had some help from the XCB folks and it looks like the problem is that
> a thread with a writing request can snipe the poll() from a thread with a
> reading request.  The attached patch fixes that issue in my case, but there is
> some concern that it won't work in certain circumstances.  Please let me know
> if this works out ok (for everyone here that is capable of building their own
> libxcb).

It seems to help in my case, although I don't have a real, specific testcase. At least World of Tanks never deadlocked for me in a couple of hours of play, while usually it would deadlock multiple times in that timeframe.

Still, this isn't something which can really be "tested" as correct, given the nature of the issue. I guess https://bugs.freedesktop.org/show_bug.cgi?id=54671 is the upstream bug?
Comment 48 Rico 2012-09-13 10:27:37 CDT
Yes, that's the upstream bug.

The patch "Fix for the libxcb poll() issue" also solves the issue for me.
Comment 49 Chiitoo 2012-09-14 04:04:03 CDT
(In reply to comment #46)
> Yeah, that seems unlikely...  Maybe t800 can help you test the patch?  I'm not
> very familiar with Gentoo, so I'm afraid I would not be very helpful.

Well, I am quite confident that I get it patched correctly (though I admit I can not bee 100% certain).  While trying to think of ways to confirm, I noticed the test case at:

https://bugs.freedesktop.org/show_bug.cgi?id=54671#c12

Being still very much a noob with programming, it took a while to get that compiled.  But I think (while uncertain) that I did figure it out, eventually.  ^^;

When I run it without the patch, it produces only the 'o', and with the patch, both the 'o' and the 'x'.  

Is this right?

Curiously, Konsole (and the output) will freeze when/if moving the mouse over it during that time.  Also switching to the tab after it's stopped it lags for a moment.  Guess it does not enjoy too many of the o and/or the x at the same time...
Comment 50 Chiitoo 2012-09-14 04:19:05 CDT
Ack, forgot to add that back traces during the freeze are very much alike the one at:

https://bugs.freedesktop.org/show_bug.cgi?id=54671

And a WINEDEBUG=relay will (sometimes) produce the following non-stop:

0025:Call KERNEL32.Sleep(0000000a) ret=0040a778
0025:Ret  KERNEL32.Sleep() retval=00000000 ret=0040a778
0025:Call user32.PeekMessageW(011be9b4,00000000,00000000,00000000,00000001) ret=0040a6c8
0025:Ret  user32.PeekMessageW() retval=00000000 ret=0040a6c8
0025:Call user32.IsWindowVisible(0001006a) ret=0040a74b
0025:Ret  user32.IsWindowVisible() retval=00000000 ret=0040a74b

But indeed, it may hang at a different spot so it doesn't always end up with this flood.
Comment 51 Chadwick Ferguson 2012-09-14 04:40:34 CDT
(In reply to comment #46)
> (In reply to comment #45)
> > Hmmm hrm, does not seem to affect me (or rather PlayOnline Viewer).  Perhaps
> > the issue in my case is something different after all, though it is peculiar
> > that the same commit would affect it in rather similar way(s)...
> 
> Yeah, that seems unlikely...  Maybe t800 can help you test the patch?  I'm not
> very familiar with Gentoo, so I'm afraid I would not be very helpful.

Sorry to burst bubbles... both your patches applied, 1 to libxcb 1.8.1 and the other to wine git 

err:ntdll:RtlpWaitForCriticalSection section 0x7e3229a0 "wined3d_main.c: wined3d_cs" wait timed out in thread 0026, blocked by 003a, retrying (60 sec)
Comment 52 Chadwick Ferguson 2012-09-14 04:44:47 CDT
(In reply to comment #51)
> (In reply to comment #46)
> > (In reply to comment #45)
> > > Hmmm hrm, does not seem to affect me (or rather PlayOnline Viewer).  Perhaps
> > > the issue in my case is something different after all, though it is peculiar
> > > that the same commit would affect it in rather similar way(s)...
> > 
> > Yeah, that seems unlikely...  Maybe t800 can help you test the patch?  I'm not
> > very familiar with Gentoo, so I'm afraid I would not be very helpful.
> 
> Sorry to burst bubbles... both your patches applied, 1 to libxcb 1.8.1 and the
> other to wine git 
> 
> err:ntdll:RtlpWaitForCriticalSection section 0x7e3229a0 "wined3d_main.c:
> wined3d_cs" wait timed out in thread 0026, blocked by 003a, retrying (60 sec)

http://bugs.winehq.org/show_bug.cgi?id=31578 which is why I'm here
Comment 53 t800 2012-09-14 07:48:23 CDT

(In reply to comment #49)
> (In reply to comment #46)
> > Yeah, that seems unlikely...  Maybe t800 can help you test the patch?  I'm not
> > very familiar with Gentoo, so I'm afraid I would not be very helpful.
> 
> Well, I am quite confident that I get it patched correctly (though I admit I
> can not bee 100% certain).  While trying to think of ways to confirm, I noticed
> the test case at:
> 
> https://bugs.freedesktop.org/show_bug.cgi?id=54671#c12
> 
> Being still very much a noob with programming, it took a while to get that
> compiled.  But I think (while uncertain) that I did figure it out, eventually. 
> ^^;
> 
> When I run it without the patch, it produces only the 'o', and with the patch,
> both the 'o' and the 'x'.  
> 
> Is this right?
> 
> Curiously, Konsole (and the output) will freeze when/if moving the mouse over
> it during that time.  Also switching to the tab after it's stopped it lags for
> a moment.  Guess it does not enjoy too many of the o and/or the x at the same
> time...

I applied these patches by adding them to ebuilds of app-emulation/wine and x11-libs/libxcb. Of cource I put these ebuilds to my local overlay.
Comment 54 Chiitoo 2012-09-14 09:25:35 CDT
(In reply to comment #53)
> I applied these patches by adding them to ebuilds of app-emulation/wine and
> x11-libs/libxcb. Of cource I put these ebuilds to my local overlay.

All right, yeah, that's how I did it.  Except when you mentioned patching Wine, I looked into the patch for that and wasn't sure if I had actually patched that at all!  Shouldn't be doing this while so tired!

I remember looking at it a bit at some point, but now I'm really not sure if I actually did it or not.  Either way, I did it now, but it would still seem to make no difference for me.

However, being very sleepy again I may very well be missing something... so I'll have to look into it all more at a later time.

Thank you!
Comment 55 Erich E. Hoover 2012-09-14 10:50:20 CDT
(In reply to comment #51)
> ...
> Sorry to burst bubbles... both your patches applied, 1 to libxcb 1.8.1 and the
> other to wine git 
> 
> err:ntdll:RtlpWaitForCriticalSection section 0x7e3229a0 "wined3d_main.c:
> wined3d_cs" wait timed out in thread 0026, blocked by 003a, retrying (60 sec)

Well, it's possible that we're dealing with more than one bug and that the graphics driver you're using makes a bgi difference.  I'm using the nVidia proprietary driver, you?

I hate to ask you to do this, because it's extremely painful, but would it be possible for you to revert to the commit just prior to 8a95eb060449e93bc38af1cabc928f0a959e9e14 (git reset --hard 8a95eb060449e93bc38af1cabc928f0a959e9e14^) and then try removing the X11 locks one at a time until you find the one that causes the application to lock up?  If that's not something within your skill level then that's alright, but it would be really helpful to know which X call causes the failure.
Comment 56 Chadwick Ferguson 2012-09-14 11:36:28 CDT
(In reply to comment #55)
> (In reply to comment #51)
> > ...
> > Sorry to burst bubbles... both your patches applied, 1 to libxcb 1.8.1 and the
> > other to wine git 
> > 
> > err:ntdll:RtlpWaitForCriticalSection section 0x7e3229a0 "wined3d_main.c:
> > wined3d_cs" wait timed out in thread 0026, blocked by 003a, retrying (60 sec)
> 
> Well, it's possible that we're dealing with more than one bug and that the
> graphics driver you're using makes a bgi difference.  I'm using the nVidia
> proprietary driver, you?
> 
> I hate to ask you to do this, because it's extremely painful, but would it be
> possible for you to revert to the commit just prior to
> 8a95eb060449e93bc38af1cabc928f0a959e9e14 (git reset --hard
> 8a95eb060449e93bc38af1cabc928f0a959e9e14^) and then try removing the X11 locks
> one at a time until you find the one that causes the application to lock up? 
> If that's not something within your skill level then that's alright, but it
> would be really helpful to know which X call causes the failure.

Nvidia 295 latest.  Its a tricky beast to get to hang its not 100% and game sometimes can go 30 mins+
Comment 57 Erich E. Hoover 2012-09-14 11:42:40 CDT
(In reply to comment #56)
> ...
> Nvidia 295 latest.  Its a tricky beast to get to hang its not 100% and game
> sometimes can go 30 mins+

Would you mind using winedbg to do a backtrace on the "blocked by" thread?  Maybe it's not locking up in the same routine we've been exploring.
Comment 58 Rico 2012-09-14 12:18:23 CDT
I tried Sacred Underworld (2.28) only with the xcb patch and it didn't crash
within 3 hours. Well then I gave up testing.

While trying without the xcb patch it crashed on several tries <10min.

If you don't know if you are running the correct patched libxcb, there are
several ways to verify it. The easiest would be to add a printf with a static
variable, so that it would print "hello, this is the patched libxcb".
Otherwise, you may have a look into /proc/<pid>/maps which lib is used.
Comment 59 Chadwick Ferguson 2012-09-16 02:08:40 CDT
(In reply to comment #57)
> (In reply to comment #56)
> > ...
> > Nvidia 295 latest.  Its a tricky beast to get to hang its not 100% and game
> > sometimes can go 30 mins+
> 
> Would you mind using winedbg to do a backtrace on the "blocked by" thread? 
> Maybe it's not locking up in the same routine we've been exploring.

Sorry to keep you hanging I did a bt but it went back to xcb.  Then I remembered that *buntu is a groovy multiarch library system so I may have not had the patched libxcb:i386 installed

I am retrying the build tonight will have more tomorrow.  Sorry if this turns out to be a false positive.
Comment 60 MonoBOY 2012-09-16 22:56:12 CDT
wine 1.5.13 , libxcb-1.8.1 , gentoo , bordarlands freezes
Comment 61 MonoBOY 2012-09-17 00:10:47 CDT
wine 1.5.13 , libxcb-1.8.1 , gentoo , bordarlands freezes
Comment 62 Chadwick Ferguson 2012-09-17 03:10:16 CDT
Created attachment 41705 [details]
Backtrace crash but no hang reported

The game froze here
Comment 63 Chadwick Ferguson 2012-09-17 03:11:03 CDT
Created attachment 41706 [details]
Backtrace crash but no hang reported

Game froze here too
Comment 64 MonoBOY 2012-09-17 03:15:28 CDT
I put the patch, but it did not help
Comment 65 Rico 2012-09-17 04:27:37 CDT
Created attachment 41708 [details]
Fix for the libxcb poll() issue - debug version

Attached is a debug patch for libxcb. It should print "Patched: libxcb poll debug patch (!c->in.reading)" when it was applied and used correctly.
Comment 66 MonoBOY 2012-09-17 04:45:07 CDT
ebuild '/usr/portage/x11-libs/libxcb/libxcb-1.8.1.ebuild' unpack
patch -p1 < /tmp/portage/x11-libs/libxcb-1.8.1/work/libxcb-1.8.1/patch.diff
ebuild '/usr/portage/x11-libs/libxcb/libxcb-1.8.1.ebuild' compile
ebuild '/usr/portage/x11-libs/libxcb/libxcb-1.8.1.ebuild' install
ebuild '/usr/portage/x11-libs/libxcb/libxcb-1.8.1.ebuild' qmerge
after this borderland did not work and freezes any
Comment 67 Rico 2012-09-17 04:50:11 CDT
I opened a thread in wine forum (http://forum.winehq.org/viewtopic.php?f=2&t=17203). Please use this thread for the discussion if you have some problems with patching, verifying if the patch was applied correctly and so on.
Comment 68 MonoBOY 2012-09-17 04:57:58 CDT
(In reply to comment #67)
> I opened a thread in wine forum
> (http://forum.winehq.org/viewtopic.php?f=2&t=17203). Please use this thread for
> the discussion if you have some problems with patching, verifying if the patch
> was applied correctly and so on.

emerge --help,  and read pls
Comment 69 Rico 2012-09-17 05:19:48 CDT
Comment #67 was meant as general suggestion, not specifically to your comment.

@MonoBOY
Bugzilla is not something to tackle user problems. It's used for technical background. I'm not sure what your problem is? Your comments make no sense at all. You should probably describe what you are trying to do. Distro specific issues don't belong here. Read your user manual if you have problems.

>wine 1.5.13 , libxcb-1.8.1 , gentoo , bordarlands freezes

>I put the patch, but it did not help

>after this borderland did not work and freezes any

>emerge --help,  and read pls

Well for me it looks like it always freezes for you. So what's your problem? The problem before and after the "patch" seems to be the same for you. Did you apply it correctly? How did you verify this? You've nowhere said which patch you use at all. There is no way to help you with the amount of information you put up here. Regardless your input seems not very useful for the bug anyway. If you have questions what you should write here to help, please use the forum.
Comment 70 MonoBOY 2012-09-17 05:30:20 CDT
(In reply to comment #69)
> Comment #67 was meant as general suggestion, not specifically to your comment.
> 
> @MonoBOY
> Bugzilla is not something to tackle user problems. It's used for technical
> background. I'm not sure what your problem is? Your comments make no sense at
> all. You should probably describe what you are trying to do. Distro specific
> issues don't belong here. Read your user manual if you have problems.
> 
> >wine 1.5.13 , libxcb-1.8.1 , gentoo , bordarlands freezes
> 
> >I put the patch, but it did not help
> 
> >after this borderland did not work and freezes any
> 
> >emerge --help,  and read pls
> 
> Well for me it looks like it always freezes for you. So what's your problem?
> The problem before and after the "patch" seems to be the same for you. Did you
> apply it correctly? How did you verify this? You've nowhere said which patch
> you use at all. There is no way to help you with the amount of information you
> put up here. Regardless your input seems not very useful for the bug anyway. If
> you have questions what you should write here to help, please use the forum.

its enougth?
[ebuild   R    ] x11-libs/libxcb-1.8.1  USE="-doc (-selinux) -static-libs" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB


>>> Verifying ebuild manifests

>>> Emerging (1 of 1) x11-libs/libxcb-1.8.1
 * libxcb-1.8.1.tar.bz2 SHA256 SHA512 WHIRLPOOL size ;-) ...                                        [ ok ]
>>> Unpacking source...
>>> Unpacking libxcb-1.8.1.tar.bz2 to /tmp/portage/x11-libs/libxcb-1.8.1/work
>>> Source unpacked in /tmp/portage/x11-libs/libxcb-1.8.1/work
>>> Preparing source in /tmp/portage/x11-libs/libxcb-1.8.1/work/libxcb-1.8.1 ...
 * Applying user patches from /etc/portage/patches//x11-libs/libxcb ...
 *   libxcb.patch ...                                                                               [ ok ]
Comment 71 MonoBOY 2012-09-18 01:47:08 CDT
Created attachment 41723 [details]
Borderlands, steam, freez

freez
Comment 72 ALeX 2012-09-21 07:50:26 CDT
Confirm bug with World of Tanks. Sometime freezes on load location with output to console:
err:ntdll:RtlpWaitForCriticalSection section 0x7dff89b8 "/var/tmp/portage/app-emulation/wine-1.5.13/work/wine-1.5.13/dlls/wined3d/wined3d_main.c: wined3d_cs" wait timed out in thread 0009, blocked by 002d, retrying (60 sec)
Applying "Fix for the libxcb poll() issue" and "Add the X11/GL lock back around XCheckIfEvent" patches give no effect.

Linux Gentoo x86_64
x11-libs/libxcb-1.8.1
app-emulation/wine-1.5.13
x11-drivers/nvidia-drivers-295.71
Comment 73 Martti 2012-09-21 08:49:17 CDT
I think same issue affects even Borderland 2. I've got basically same Gentoo config as ALeX and I'm getting freezes in both Bordelands 1 & 2
Comment 74 t800 2012-09-21 17:35:49 CDT

I don't confirm freez for Borderlands but it happen with Mass Effect 3 sometimes.

Linux Gentoo x86_64
x11-libs/libxcb-1.8.1
app-emulation/wine-1.5.13
x11-drivers/nvidia-drivers-295.71
NVIDIA GT120
Comment 75 John P Sims 2012-09-21 21:21:54 CDT
Rico, Erich,

I followed the instructions in the forum.

 http://forum.winehq.org/viewtopic.php?f=2&t=17203

Borderlands no longer freezes in either original scenario. I haven't played extensively, but the original issues seem to be resolved with the patched libxcb from git and wine-1.5.13-232-gc0d2029 (also from today's git).

Well done, gentlemen. Thank you.

So where do we go from here?
Comment 76 Erich E. Hoover 2012-09-22 17:39:48 CDT
(In reply to comment #75)
> Rico, Erich,
> ...
> So where do we go from here?

We wait for the XCB folks to review the problem and the suggested patch.  Unfortunately, we have pretty bad timing here since the XDC2012 was this last week.
Comment 77 Erich E. Hoover 2012-09-30 17:37:31 CDT
This issue is now resolved upstream by XCB commit 23911a707b8845bff52cd7853fc5d59fb0823cef.  I'm working with the Ubuntu folks (and trying to contact the Debian folks) to get this commit included in the repositories ASAP.  Please update to the latest git for libxcb and open a new bug if you find any further issues.
Comment 78 Chiitoo 2012-10-01 05:08:29 CDT
Hurray~

I shall see if I can get it to Portage, after testing it myself.


Thank you (everyone) for all your efforts!
Comment 79 ALeX 2012-10-01 12:01:26 CDT
World of Tanks 0.8.0 still affected. Used libxcb-1.8.1 with patch from 23911a707b8845bff52cd7853fc5d59fb0823cef commit.
Comment 80 Erich E. Hoover 2012-10-01 12:34:48 CDT
(In reply to comment #79)
> World of Tanks 0.8.0 still affected. Used libxcb-1.8.1 with patch from
> 23911a707b8845bff52cd7853fc5d59fb0823cef commit.

Did any of the previous patches help (the commit is slightly different from what we were testing before)?  Alexandre noticed a potential lockup that's a different issue, so if the other patch attempts don't help then I'd say it's worth opening another bug.
Comment 81 ALeX 2012-10-01 13:24:38 CDT
(In reply to comment #80)
> (In reply to comment #79)
> > World of Tanks 0.8.0 still affected. Used libxcb-1.8.1 with patch from
> > 23911a707b8845bff52cd7853fc5d59fb0823cef commit.
> 
> Did any of the previous patches help (the commit is slightly different from
> what we were testing before)?  Alexandre noticed a potential lockup that's a
> different issue, so if the other patch attempts don't help then I'd say it's
> worth opening another bug.

None of the patch does not help. The same error as before (comment 72). Open a new bug.
Comment 82 Erich E. Hoover 2012-10-01 14:57:49 CDT
(In reply to comment #77)
> This issue is now resolved upstream by XCB commit
> 23911a707b8845bff52cd7853fc5d59fb0823cef.  I'm working with the Ubuntu folks
> (and trying to contact the Debian folks) to get this commit included in the
> repositories ASAP.  Please update to the latest git for libxcb and open a new
> bug if you find any further issues.

Robert Hooker has kindly prepared testing packages for Ubuntu 12.04 and 12.10 at this PPA:
https://launchpad.net/~sarvatt/+archive/sru8
Comment 83 Forest 2012-10-01 20:14:42 CDT
Launchpad is telling me there's a newer version available in Quantal:

https://launchpad.net/ubuntu/quantal/+source/libxcb/1.8.1-1ubuntu1

The changelog says it includes the fix, probably in the nick of time considering that Quantal is going to be released soon.  Nice work, all!  I hope it fixes bug 31578.
Comment 84 Chiitoo 2012-10-02 07:30:24 CDT
Meh, it's not the first time I forget to think about the 'emul-linux' packages...  Oh well, in any case, an explanation for the Gentooers appears:


https://bugs.gentoo.org/show_bug.cgi?id=436780#c4


Tiziano Müller   2012-10-02 14:36:02 EEST 
The problem here is that wine on amd64 uses the libxcb from emul-linux-x86-xlibs in /usr/lib32 while the libxcb installs only the libxcb in /usr/lib64

But I can confirm that the patch mentioned fixes deadlock issues with Dragon Age 1 and 2.


So if I understand it correctly, you will be experiencing the issue until the emul-linux package is updated.  Until then, one could simply build (but not install) libxcb from git, and use it from there.  I see no harm in that.

One may simply follow the directions here:

http://forum.winehq.org/viewtopic.php?f=2&t=17203

Build a 'stand-alone' version of libxcb from git.

One may skip the parts that deal with the patch, and run the application(s) as described in the guide (add the LD_LIBRARY_PATH into a short-cut to the game for example).


Thank you again, Tiziano Müller, et al.
I hope this helps~
Comment 85 Erich E. Hoover 2012-10-02 10:18:02 CDT
(In reply to comment #83)
> Launchpad is telling me there's a newer version available in Quantal:
> 
> https://launchpad.net/ubuntu/quantal/+source/libxcb/1.8.1-1ubuntu1
> 
> The changelog says it includes the fix, probably in the nick of time
> considering that Quantal is going to be released soon.  Nice work, all!  I hope
> it fixes bug 31578.

I was trying my best to push this so that it would make it time, glad to see that it did.  The bug has been nominated for inclusion in precise as well, so we will hopefully see this fixed for the vast majority of users soon.

Please let us know if the package fixes it either way, it sounds like there is an additional bug that affects some apps and not others.
Comment 86 Rosanne DiMesio 2012-10-07 14:17:16 CDT
*** Bug 31901 has been marked as a duplicate of this bug. ***
Comment 87 Roman 2012-10-08 06:05:22 CDT
What about bug 31849 --> http://bugs.winehq.org/show_bug.cgi?id=31849
Same as this bug, but in World of Tanks this pathes give no effect.
Comment 88 Rico 2012-10-09 01:38:12 CDT
*** Bug 31849 has been marked as a duplicate of this bug. ***
Comment 89 Alexandre Julliard 2012-10-10 11:20:14 CDT
*** Bug 31883 has been marked as a duplicate of this bug. ***
Comment 90 Rico 2012-10-16 08:31:44 CDT
*** Bug 31943 has been marked as a duplicate of this bug. ***
Comment 91 BitMaster 2012-10-26 17:31:40 CDT
*** Bug 32014 has been marked as a duplicate of this bug. ***
Comment 92 Henri Verbeet 2012-11-03 05:54:12 CDT
Closing.
Comment 93 Dmitry Eliseev 2013-01-22 06:11:18 CST
*** Bug 32607 has been marked as a duplicate of this bug. ***


Hosted By CodeWeavers