Answering myself:
Firmware for Volataire 500 EX-D can be found here: Firmware for ConnectX® - Mellanox Technologies. Look for MHEH28-XTC (PSID: MT_04A0110001)
The problem with "Modify QP error" and osm.log flooding persists even after upgrading to firmware 2.9.100 and seems related to OpenSM when using dual port end-to-end connections with two cables.
--
I can't figure out what firmware to use for MT_04A0110001 (Voltaire 500 EX-D dual port). The information of the device is listed as:
Image type: FS2
FW Version: 2.7.700
Device ID: 25408
Description: Node Port1 Port2 Sys image
GUIDs: 0008f104039a1ea5 0008f104039a1ea6 0008f104039a1ea7 0008f104039a1ea8
MACs: 000000000000 000000000001
VSD: VLT-HRM
PSID: MT_04A0110001
The reason why I’m trying to upgrade the firmware is to check if that will fix the following error that appears when opensm (from MLNX_VPI_WinOF-4_80_All_win2012R2_x64.exe) is started on windows server 2012 r2 :
"Mellanox ConnectX IPoIB Adapter device reports a "Modify QP error" on qpn #0x20b8 Status #0xffffffea. Therefore, the HCA Nic will be reset. (The issue is reported in Function CMcast::CompleteJoinMcastWi)." (This message is flooding the windows server event log several times a second...)
Also, %windir%\temp\osm.log is flooded with thousands of lines repeating the same thing:
log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:401b:ffff::ffff:ffff
log_notice: Reporting Generic Notice type:3 num:67 (Mcast group deleted) from LID:1 GID:ff12:401b:ffff::ffff:ffff
log_notice: Reporting Generic Notice type:3 num:66 (New mcast group created) from LID:1 GID:ff12:401b:ffff::ffff:ffff
log_notice: Reporting Generic Notice type:3 num:67 (Mcast group deleted) from LID:1 GID:ff12:401b:ffff::ffff:ffff
--
Could this may be an issue related to OpenSM only?
Thanks in advance!
Lars.
Maybe this will help: Deploying Windows Server 2012 with SMB Direct (SMB over RDMA) and the Mellanox ConnectX-2/ConnectX-3 using InfiniBand.
Check the PowerShell commands from section 6 (i.e. "Verify everything is working") and onwards. Please keep us posted how things are working out...
--
Thanks in advance and good luck!
Thank you for your reply.
I know this document well. I have set my server and clients up according to it. I have flow-control setup on my extreme 670 switch and the ports tagged.
I just cannot establish SMB-Multichannel connection between client and server. Clients and server all show RDMA capable in the get-smbmultichannelconnection ps command.
The only lead I had was the perf counters for Mellanox Adapter Diagnostic was show Responder QCE errors.
Whats weird is that on the same client the SMB Direct and the RDMA activity perfmon counters show a lot of activity.
I just need to verify that RDMA will work reliably under load in production.
Thanks Again
Hi Igor,
Please sent the output of the command below on your server:
# dkms status -m "mlnx-ofed-kernel" -v "2.3"
Did you try to run the installation on other servers with the same OS?
Thanks,
Vladimir
Hi Vladimir,
same result on other server with same OS and identical HW.
Provided "dkms" command returns without any output.
# dkms status -m "mlnx-ofed-kernel" -v "2.3"
#
Hi Igor,
Can you arrange a remote access to one of your servers?
I will check if that is possible, it may take time anyway.
Is there anything else I can do? Any debug flags, logs?
Hi,
Could you open a support case?
The support team will follow up from there.
Thanks,
Ophir.
Enable IPv6 support on the hosts by running command:
echo 0 > /proc/sys/net/ipv6/conf/enp131s0/disable_ipv6
ipv6 is already enabled for the hosts.
Opir, does MXM work with ROCE? Our setup at University of Houston is ROCE. I beginning to suspect that MXM is strictly meant for infiniband protocol, any taughts before I formerly engage the support Team.
-Jerry Ebalunode
MXM can use Ethernet layer, but it doesn't use RoCE.
Please, e-mail me and let's continue via e-mail.
Hi,
The download link http://www.mellanox.com/downloads/ofed/MLNX_OFED-1.5.3-3.1.0/MLNX_OFED_LINUX-1.5.3-3.1.0-rhel6.1-x86_64.tgz does not seem to work anymore.
Please help
Thanks
-Minh
I am trying to install a ConnectX-3 Pro in a BSD system. I rebuilt and installed the new kernel but when I try to build the modules I always fail the "make && make install" - for the mlx4 module I get:
/tmp/MLNX_EN_FreeBSD_v2.1/modules/mlx4/../../ofed/drivers/net/mlx4/mcg.c:43:17: error: unused variable
'zero_gid' [-Werror,-Wunused-const-variable]
static const u8 zero_gid[16]; /* automatically initialized to 0 */
^
1 error generated.
*** Error code 1
For the mlxen module I get:
/tmp/MLNX_EN_FreeBSD_v2.1/modules/mlxen/../../ofed/drivers/net/mlx4/en_netdev.c:1694:19: error: unused
variable 'fmt_u64' [-Werror,-Wunused-const-variable]
static const char fmt_u64[] = "%llu\n";
^
1 error generated.
*** Error code 1
Stop.
This is with BSD 10.1 and the "MLNX_EN_FreeBSD_v2.1" tarball.
Hello,
I was wondering if FCA or other accelerators work with RHEL6u4 or RHEL6u6 OFEDs?
Are there any issues to watch out for or is some functionality not available with the RHEL OFEDs but it is with Mellanox OFEDs?
Thanks!
Michael
Hi Vladimir,
please send me your email I will send you details how to connect to the machine.
Thanks
Hi,
The download link http://www.mellanox.com/downloads/ofed/MLNX_OFED-1.5.3-3.1.0/MLNX_OFED_LINUX-1.5.3-3.1.0-rhel6.1-x86_64.tgz does not seem to work anymore.
Please help
Thanks
-Minh
Enable IPv6 support on the hosts by running command:
echo 0 > /proc/sys/net/ipv6/conf/enp131s0/disable_ipv6