XBEE WIFI – DIGI – TCP Receive Problem
So I just bought two new XBEE WIFIs. If you’re not familure with XBEE’s they are little embedded RF Modules that provide a simple serial connection. In the past they have always been over the 802.15.4 protocol, but the new ones communicate over 802.11g/n! up to 65MBps. The new WIFI ones just came out a couple months ago, and there is little documentation online. Digi does provide a manual online.
So with XBEE’s you have Transparent AT Command Mode and API Mode. In Transparent mode you send serial ascii to both send data and must put the xbee in command mode to change any configuration items. You enter command mode by sending three consecutive +’s (“+++”). Now you can send Ascii Command’s with parameters such as “ATFR” to restart the module The more robust communication style is API Mode. In this mode you must generate XBEE API Packets at the byte level.
After configuring the XBEE in the X-CTU application to my Access Point (which it supports WPA2). It automatically DHCP’s an address.
And the problems begin…
I set the Registers for DEST_IP, DEST_PORT, SRC_PORT, PROTOCOL=TCP. After setting those registers, if I then send the module this data serial over UART it automatically creates a TCP/IP socket to the DEST_IP on SRC_PORT from DEST_PORT and sends data. It has no problem creating the Three Way Handshake, sending data over the tcp stream, or even closing the tcp stream. The problem that I have had is receiving data from the TCP Stream. In Transparent Mode it’s supposed to just come back over the UART. it’s not… It’s API Mode it’s supposed to generate a 0xB0 … it doesn’t…
Example Data Sent to XBEE serially [these are the same thing]
Transparent Mode: GET /energy/CommitEnergy.php HTTP/1.0 Host: s157582434.onlinehome.us API Mode: 7E 00 57 20 01 4A D0 3B 1A 00 50 BE EF 01 00 47 45 54 20 2F 65 6E 65 72 67 79 2F 43 6F 6D 6D 69 74 45 6E 65 72 67 79 2E 70 68 70 20 48 54 54 50 2F 31 2E 30 0D 0A 48 6F 73 74 3A 20 73 31 35 37 35 38 32 34 33 34 2E 6F 6E 6C 69 6E 65 68 6F 6D 65 2E 75 73 0D 0A 0D 0A 0D 0A 12
I don’t understand why I’m not getting an HTTP response, I know that the web server was receiving the GET Request in both modes because it saves a msg to a log file. I unsecured my wireless network.. started up BackTrack and ran WireShark in promiscuous mode. After seeing a real TCP Socket Creation and all the correct data, yet still seing no response from the XBEE I decided to email tech support. I will admit they do respond very quickly. Although it’s like they didn’t read my question or look at my pictures.
TCP Stream between XBEE to Remote Host [Looks Good!
This is an example of me sending a GET request to my webserver, and receiving no return data. I do receive a message sent successfully message. Also, If I send it more than a couple times, it actually crashes the XBEE… not sure why the socket should have closed after the HTTP OK.
Dialog with XDIGI Support
Created 10/10/2011 23:37
Background
I’m Connecting to the XBEE via an FTDI cable connected to an Adafruit XBee Adapter Kit. I’m sending API packets from X-CTU. The same problems occur in Transparent Mode. The XBEE is connected to my wireless access point and I’ve set the router to OPEN security so I could debug the traffic. It DHCP’s the IP Address of 192.168.1.100. In Wireshark I see successful connection to the DEST IP with data coming back.
Connection Setup
I’m establishing a connection to a remote web server <4AD03B1F>(74.208.59.26) to port <0050>(80) from port <BEEF>(48879).
Problem #1) I never receive any data back in either Transparent or API Mode. I’m expecting an API packet of B0 (Rx IPV4 Packet). In wireshark I have seen TCP data being returned to the XBEE. [Attached]
Problem #2) If I try the connection again even though the webserver should have already closed the TCP Connection it crashes the XBEE. I’ve even seen the FIN, ACK in wireshark. I receive
[89] 01 04 – Physical error occurred on the interface with the WIFI Transeiver.
[8A] 03 72 – No longer joined to access point.
From that point if I try to transmit I get a:
[89] 01 03 – Transmission was purged because it was attempted before stack was up.
Data Sending [Transparent & API Mode]
GET /energy/CommitEnergy.php HTTP/1.0
Host: s157582434.onlinehome.us
//SEND Get REQUEST to Web Server
7E 00 57 20 01 4A D0 3B 1A 00 50 BE EF 01 00 47 45 54 20 2F 65
6E 65 72 67 79 2F 43 6F 6D 6D 69 74 45 6E 65 72 67 79 2E 70 68
70 20 48 54 54 50 2F 31 2E 30 0D 0A 48 6F 73 74 3A 20 73 31 35
37 35 38 32 34 33 34 2E 6F 6E 6C 69 6E 65 68 6F 6D 65 2E 75 73
0D 0A 0D 0A 0D 0A 12
10/11/2011 – 08:15 (Case Set to Presumably Fixed & Closed)
Matthew,
Thank you for contacting Digi. Try entering the port numbers in the X-CTU for the WIFI module in the Hex values. I think you will have better luck.
If there is anything else I can do for you, please let me know.
Sincerely,
Eric Flanders
Digi RF Technical Support
10/11/2011 – 09:55 (My Response to this statement)
I did use hex… I used 0x0050 for port 80, and 0xBEEF for port for 48879. I see that the XBEE is making clear connections and the webserver is sending TCP data back via Wireshark AIRMON capturing. However, no data is returned in either Tranparent Mode or API Packet B0 from the XBEE.
10/11/2011 – 10:11
Matthew,
Are you able to get it to work with the X-CTU and Hyper-terminal? Are you able to enter command mode and make changes to the radios settings?
10/11/2011 – 11:10
Yes, I can use Command Mode or API Mode to change radio settings.
In Transparent Mode I cat set the DEST in HEX, and PORT in HEX.
I am making a successful TCPIP Connection!!
The Device is doing the TCP Handshake, and TCPIPv4 Sending and in API Mode I receive a Success Message. The webserver did receive the Request because it acted on the the XBEE’s Request.
However, the XBEE is not showing me that it received the webserver’s communication back! In Transparent Mode I get nothing else in X-CTU Terminal and in API Mode I get no TCPIP Receive (B0) Packet.
10/11/2011 – 13:37 (wow.. and he got a leet time…)
Matthew,
To help narrow down the possible cause, are you able to use Hyper-terminal and X-CTU to create a connection to the radio via the PC’s COM port and the WIFI connection and then send and receive data back and forth? Keep in mind that your web server can’t do this so we are trying to take it out of the picture to see if it is an issue with it or not.
10/11/2011 – 14:59 (My Response)
Yes, I can make a connection to the XBEE over COM6.
Yes, I can connect and send TCP packets in either Transparent or API over WIFI.However, if server sends data back, example:
TCP Packets returned from the webserver! through my router. (directed to my XBEE) they don’t appear
I know they are returned to the XBEE because I can see them from a third party wireshark sniffer in premiscuous mode..The problem is XBEE does not aknowledge receiving TCP Packets!
Please view images from my original email, it shows a clear TCP/IP Connection to Close from XBEE to WebServer!
While Hyper Terminal or X-CTU shows ZERO data returned.
Attached is another image.
10/11/2011 – 15:08
Matthew,
The radio does not accept direct TCP/IP packets. You need to create a socket to it for it to send and receive data via its WIFI ports.
10/11/2011 – 18:40
Ok… If I am sending data to a remote server over TCP/IP I am creating a socket correct? You use API “FrameType 0x20 Transmit (TX) request IPv4” to create a socket correct?
AFAIK it creates the socket automatically when you send data via the 0x20 command, am I mistaken? because that is what it is doing… and it creates a TCP Socket with 3-way hand shake and all, and sends the data….
The only problem is.. it’s not returning the received data from the TCPIP Socket to the ReadSerial with a 0xB0 API Packet.
And we wait for a response…
10/12/2011 – I was able to get the wifi expert on the phone and talked to him about the problem. He’ll be looking in to it.
10/18/2011 – Video I just created tonight to show problem.
*You’ll probably want to watch this in HD.
3/9/2012
I finally did get a response back from the DigiKey rep.
Welcome back from you deployment. The new version of the firmware fixes the TCP handling issues you were seeing.
When you are ready, just give me a call and we can continue to work on the XBee WF.
I did specifically test this. Works Fine — I was able to connect to a TCP server (e.g. webserver) and data received before timeout (connection closes) it returns the received data back to me as the end user.
Now on to the DNS Issues…. X_X
GodyDeeps 
Hey i’m wondering if you can help me. I’m trying to netstat the thing with no luck!
What do you mean netstat it? Netstat is a windows application that shows your current connections… the XBEE does respond to ICMP Pings.
You have having more luck than I am. Is there anyway you can explain to me how to set this thing up? In my case I’m using only one XBee Wi-Fi which I am trying to send data back and forth between PC and module (since its connected to mcu’s uart). So far the XBee connects to AP, but sides that, can’t even see a “Hello”, heck, can’t even get Putty to connect to the darn thing.
Any help would be greatly appreciated. Thanks.
I’m doing the same thing, one module to a PC. The problem that I’m having actually only applies when the XBEE is the client. If the XBEE is the server It appears to work just fine. I just attached a video to the post that shows this.
woohoo, i at least made it work, like u said, to and fro, by connecting to the Xbee’s IP, the problem is now finding if the thing can auto connect to an available wi-fi, and not the specific AP in the SSID field. or maybe an over the air connection to xctu to set without removing it from the device…
anyway, thanks to u i was able to find some of the errors in my setting (ex. fighting with port 2616 and turns out that was in hex, jaja.)
I noticed that wireshark is showing some of those packets as HTTP. I checked the xbee wifi docs and I don’t see any way to make HTTP GET/POST requests. How are you getting HTTP packets? Does the application you wrote send HTTP headers – if so it looked like you had HTTP going both ways so how is the Xbee sending HTTP requests?
Here’s why I ask: I’m trying to create a sensor network with the wifi modules. I’m attempting to connect a simple temp sensor to the xbee directly and then make periodic HTTP GET or POST requests with the data. I’d like to do all of this directly from the Xbee without another microcontroller (arduino or etc) in between.
Does this seem possible to you in principle? So far I don’t see how to accomplish this. Have I misunderstood the Xbee Wifi’s capabilities?
I understand that I could create a software listener to receive data on certain tcp ports and then convert them into the HTTP requests – is it not possible to do it straight from the XBEE?
The tcp response packet isn’t a big deal for me I don’t think? I can just process the request without ever returning anything – the Xbee doesn’t need any response data in my scenario (sucks that it doesn’t seem to be responding for you).
Let me know if the question isn’t clear. Thanks for reading.
Wireshark shows it as “HTTP” because it looks like HTTP and its on port 80..
HTTP is just a standardized format of data sent over TCP.
My Arduino is telling the XBEE to send the HTTP packets: “GET xxxx HTTP/1.0” .. the XBEE has configured the servers IP and the PORT 80..
The XBEE opens the TCP SOCKET and then just sends and receives data… (minus the receiving data correctly) You have to manage the HTTP Formatting
AFAIK, you will not be able to send the sensor data back in HTTP packets. The XBEE wifi module will not write HTTP format which look like:\
GET /path/myFile.php?value=_SENSOR_DATA_ HTTP/1.0
If you are going to have multiple sensors around, we were going to have multiple Slaves(sensors/xbees) talk to one of the modules that is a Master(sensor/arduino/xbee)… that way the master could be responsible for sending the area’s data up to a web server periodically.
You’re right, the HTTP response isn’t that important for me as well, if I’m just wanting to save data off… I was really hoping to get a response for a few reasons… #1) error handling, make sure the server received the response correctly.. #2) time sync… I was going to have the web server send the time back… my Arduino loses accuracy about 3-4 seconds every hour… which means every day its loses a minute.
However, I’m sometimes running into an issue with sending the GET request multiple times, even though the connection should have been closed as soon as the server responded and closed the connection.
Yeah I just saw your kill-a-watt project and saw that you’re using an arduino. I thought you were using the Xbee directly somehow. It’s too bad that the data from xbee inputs can’t be encapsulated in arbitrary text so as to make simple HTTP requests everything else is there. Then it seems to me very, very cheap wifi sensor networks could be built with ease 😦
Out of curiosity why do you care about time syncing the arduino? Can’t the data be time stamped as it’s received from the wifi? It might be slightly off real-time but it should be in terms of ms – whatever the network latency is.
Error correction would still be a problem…
Anyway I like your project and I’m tracking the progress – keep us updated! If you get this up and running I’ll be building one right behind you. I’d be interested in how you’re going to power it and enclose it for long-term use.
I want to use the time syncing because I want to capture the sensor data which happens every 1-3 seconds… I don’t want to send an HTTP request that often… I can save power by only waking the WIFI every 1-3 minutes and sending the past 100 sensor values as/with timestamps.
error correction such as collisions and radio interference should still be handled at the TCP stack and down.
As for power and enclosure, I’m lucky. My garage has a side door, and the meter is a foot beside that, for my house, it’s going to run from a cell phone charger, and then 3 wires are going to go outside (2-3 feet) that connect to a dime-sized IR Recvr. However, I would like to work on a battery powered version that uses sleep and more energy efficientmethods, and possibly power from a solar panel.