OG 1.06 web server went unresponsive this morning

[schwim]

Hi…

I’ve been using the OG to control a roll-off roof on an astronomical observatory for about 2 years now. It’s worked perfectly all along until today.

At 9:30AM I got an automated alert that the control systems could no longer see the OG. I investigated and found that I can ping it, connect to TCP/80, but it will not respond to any HTTP requests at all. I’ve power cycled it about a dozen times. In 2 cases it did come back and respond for about a minute, then went unresponsive again.

One thing I have limiting me from further troubleshooting is the fact that this location is 180 miles away and I do not have easy physical access to try a factory reset.

Is there anything I can do to try to recover this without making a day of it by going on site?

[schwim]

In the course of troubleshooting I noticed the OG keeps asking for a DHCP address every 15 seconds or so. Not sure if this is indicative of a boot loop.

Jan 11 11:37:27 observatory dnsmasq-dhcp[5406]: DHCPDISCOVER(ap01) 5c:cf:7f:d6:1c:af
Jan 11 11:37:27 observatory dnsmasq-dhcp[5406]: DHCPOFFER(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:37:27 observatory dnsmasq-dhcp[5406]: DHCPREQUEST(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:37:27 observatory dnsmasq-dhcp[5406]: DHCPACK(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af ESP_D61CAF
Jan 11 11:37:41 observatory dnsmasq-dhcp[5406]: DHCPDISCOVER(ap01) 5c:cf:7f:d6:1c:af
Jan 11 11:37:41 observatory dnsmasq-dhcp[5406]: DHCPOFFER(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:37:41 observatory dnsmasq-dhcp[5406]: DHCPREQUEST(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:37:41 observatory dnsmasq-dhcp[5406]: DHCPACK(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af ESP_D61CAF
Jan 11 11:37:56 observatory dnsmasq-dhcp[5406]: DHCPDISCOVER(ap01) 5c:cf:7f:d6:1c:af
Jan 11 11:37:56 observatory dnsmasq-dhcp[5406]: DHCPOFFER(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:37:56 observatory dnsmasq-dhcp[5406]: DHCPREQUEST(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:37:56 observatory dnsmasq-dhcp[5406]: DHCPACK(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af ESP_D61CAF
Jan 11 11:38:11 observatory dnsmasq-dhcp[5406]: DHCPDISCOVER(ap01) 5c:cf:7f:d6:1c:af
Jan 11 11:38:11 observatory dnsmasq-dhcp[5406]: DHCPOFFER(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:38:11 observatory dnsmasq-dhcp[5406]: DHCPREQUEST(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:38:11 observatory dnsmasq-dhcp[5406]: DHCPACK(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af ESP_D61CAF
Jan 11 11:38:25 observatory dnsmasq-dhcp[5406]: DHCPDISCOVER(ap01) 5c:cf:7f:d6:1c:af
Jan 11 11:38:25 observatory dnsmasq-dhcp[5406]: DHCPOFFER(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:38:25 observatory dnsmasq-dhcp[5406]: DHCPREQUEST(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af
Jan 11 11:38:25 observatory dnsmasq-dhcp[5406]: DHCPACK(ap01) 172.16.255.1 5c:cf:7f:d6:1c:af ESP_D61CAF

• This reply was modified 5 years, 3 months ago by schwim.

[schwim]

Interesting find – the OG is still trying to report status to Blynk. I can see the TCP handshake and a small bit of data going out. Unfortunately I deleted that profile from my Blynk account long ago because I didn’t need it at the time. Guess I kept the key in the OG though.

Any chance this can be exploited locally to take control of it? I can spoof the blynk IP and run a flask server to issue commands. No idea what command it to do though.

[schwim]

I managed to get someone to head over and reset it for me. All is working now.

Interesting that it got in that state. I power cycled it about 30 times and only managed to get the UI up 3-4 times. Is this something that may have been identified and fixed in the later firmware versions?

[Ray]

Not sure about the cause, but it’s possible that software reboot might not be able to fix it and only a hard reboot (unplug and re-plug power) will recover the state.

[schwim]

It just happened again. Unfortunately a hard restart (power cycle) does not recover it. Same behavior as above. It’s online for about 6 seconds then reboots itself.

Any suggestions before I send someone over to poke it? It would be neat to recover it somehow.

[Ray]

You can either submit a support ticket or before that maybe try to update to firmware 1.1.0 and see if it helps.

[Author]

Posts