PIER-14 Spanish wet season for subscription piers - poll
By Asatron on Sunday, 05 December 2021
Posted in Syndicate Lounge
Replies 25
Likes 0
Views 671
Votes 0
There are a number of remote observatory companies in Spain apart from Roboscopes and overall we are very similar in how and what we do, one small difference between us however is when humidity is very high we do not close the roof. We still gather data rather than simply close the roof. The man reason for this as UK astronomers is dew and humidity is something we deal with 12 months a year and in the UK and if we closed the roof for humidity we would never open the roof a single night in all 365 of them. So we firmly believe that some data is far better than no data. Others amongst you may disagree with this philosophy and that's the point of this post This approach mind does come benefits and issues Let’s start with issues
  • Not all subs are good[/*]
  • The user has more work to sorting through and removing below par subs[/*]
  • Overall the data is not as good as it is the rest of the year due to atmospherics[/*]
  • You get all the data warts and all so yes some manual labour is 100% required[/*]
Benefits
  • The roof is open so you get data you would never receive with a closed roof[/*]
For the staff at Roboscopes the wet seasons are far more intensive for us work wise because we have to keep on top of High humidity, dew, high haze, not to mention sub standard data where we simply have to just resubmit it for the group as a whole. Every data set get checked with blink and we monitor the weather very closely this time of year because high haze has a tendency of slipping past the IR based weather systems because it may only alter the sky temperature but 1º above clear and for the most part we leave a 3º differential so the roof is not bouncing open/closed all night like a yoyo. Roboscopes staff will always do our best to keep on top of things and let you know when we have foibles etc however you must also take into account remote diagnosis of issues takes time unlike popping out into your backgarden/yard with mark1 eyeball. So at present we are in a quandary, do we mirror all the other remote observatory operators we know and close the roof when the humidity is high or do we leave it open and keep collecting data ? Obviously for us closing the roof is by far the easier option but in the very beginning Roboscopes chose not to do this as in our opinion even with the extra work involved for us and you the members, you can still get some good data/images during the wet season. I know some of you are long standing members and will remember last season was very extending rain wise and the roof remained firmly shut for long periods. This season seams to be high haze and humidity instead so far. However we feel the time has come to draw a line under it being our decision to make alone and as the members we want your opinion. So a vote from each member would be fantastic please High humidity times 1 - Roof open and keep imaging despite it being extra work for all concerned etc etc? Or 2 - Close the roof until the nudity and other atmospherics get to a lower level Whichever option gets the most votes on each pier we will simply go with what you ask Ray, Mike and other dual subscribers may we be so bold as to presume that your vote would count on any pier you are a subscriber ? Foot note: I will say that this year has been the worst of 5 years I have been in Spain, Pier 3 as well as some others we have never had to even turn the due control on in previous years so its been a little bit of an eye opener, Not to mention the complexities of getting extra items stuck in Spanish customs in this wonderful post Brexit world we now live in  Extra dew control was posted out weeks ago and fingers crossed it should land very soon. However if we don’t get it right or we mess up then you have our apologies but believe me it’s not for the want of hard work at our end but we are only human as dedicated as we are Steve and the team
Hi Steve,  Having gone through last winter season when the conditions for a while were better here in the UK, I'd also had the benefit of the prior seasons, so was able to take a measured view. For anyone who may have only recently joined it must be somewhat frustrating. Those who have been members for longer probably have some catching up to do on the data collected during the better months when it's often difficult to keep up with the data being collected. Hopefully for those most affected right now the disappointment resulting from this current run of bad data will in time dissipate. There is the prospect of amassing many hundreds of hours of data for a reasonable cost by the time the year is up.  I believe that you may have already been taking some mitigation measures Steve by limiting the imaging hours when dew is forecast, is this maybe another solution, or do you just want us to pick from the 2 options? I would prefer to stay with the current setup, option 1, and leave it to us to request replacement data. Hopefully I may have the casting vote on a couple of the piers.   Thanks for the info and suggestion.  Cheers,  Ray 
NoMoreCloudyNights
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Steve,  Having gone through last winter season when the conditions for a while were better here in the UK, I'd also had the benefit of the prior seasons, so was able to take a measured view. For anyone who may have only recently joined it must be somewhat frustrating. Those who have been members for longer probably have some catching up to do on the data collected during the better months when it's often difficult to keep up with the data being collected. Hopefully for those most affected right now the disappointment resulting from this current run of bad data will in time dissipate. There is the prospect of amassing many hundreds of hours of data for a reasonable cost by the time the year is up.  I believe that you may have already been taking some mitigation measures Steve by limiting the imaging hours when dew is forecast, is this maybe another solution, or do you just want us to pick from the 2 options? I would prefer to stay with the current setup, option 1, and leave it to us to request replacement data. Hopefully I may have the casting vote on a couple of the piers.   Thanks for the info and suggestion.  Cheers,  Ray 
Morning and Agreed Ray, Parts of last Autumn/Spring season was not much fun either hence why I proposed a poll. We love what we do at Roboscopes as you are well aware but we also have lives and need to keep a certain work life balance. We cannot do much more than monitor the skies during waking hours this time of year and keep a close eye on the humidity levels. Sometimes I feel like the BBC weather man I watch so much Radar I understand why most other remote observatories simply close the roof as it becomes hard work for us. When the Skies are good in Spain the rest of the year you get more quality data than the average person can handle but it does not seem right to what is in my mind the potential for 40-60% decent data from what we take each night this time of year. Especially what is taken earlier in the evening before lets say 11-12pm as a random time. We run the syndicates in a very different way to the hourly system and to the most part we treat the pier as "yours" so you can input your own requests, cancellations and checking of subs etc much like you would at home. This way apart from the scope being remote you really are in charge of the data, image quality and final processing just like at home. Yes we blink the data before sending over and if anything obvious jumps out we resubmit it but on the whole we leave you to it to assess the data and retake if needed just like it should be as its "your pier" From my point of view I like to see you all autonomous and discussing things and resubmitting as needed, not just from a logistical point of view for Roboscopes but because it means discussion and a true syndicate approach where you work together with us to get the best results from the Pier Nothing is perfect and we have hiccups, problems and even equipment deaths (Pier-1 Camera 6 months ago, Pier-13 Atlas focuser only last week) but working together we are trying to make this the best experience possible at a very reasonable price point. Yes we could do way way more from our end I am sure of that but then the price will increase to suit the man hours and again we don't want that, we want remote imaging to stay as affordable as possible for our users. Like I have said many times, we prefer the roof open even with high humidity even if it means "you" the users have more checking to do. However if it means undue hassle for high humidity during the wet season about data quality then we could at 75/80% humidity auto close the roof. I will leave it up to the users in each group to make a decision that suits them but from a personal point of view I hope you all keep the roof open as we will always do our best to monitor as well as adjust the weather system to close the roof if conditions are just unusable (hoping this last part removes your option 3 question Ray) So it's a simple 1 or 2 HTH Steve
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
It never occurred to me that verbal diarrhoea might be contagious Steve, but I wonder if you may have inadvertently caught it quite recently from someone at Kelling.   With my vote already cast for option 1, can I assume that voting has now closed on a couple of the syndicate piers at least. Now a couple of likely irrelevant questions, my speciality, but in the event of a tie I assume you will have the final say and when does voting close.  Cheers,  Ray 
NoMoreCloudyNights
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
I chose the full and complete answer so all users could see "Roboscopes" reasoning on the subject not just for current users but future users also, as the forum is available for everyone to read. So I went for completeness so please excuse my hyperbole on this occasion I am hoping that common sense prevails and the roof stays open so that kid off says where any casting votes may lean Steve Option 1 it is then for Ray
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi  all  i m for option 1 , i prefer data than no data (as now in my locations , 4weeks cloudy time , i m highly depressed ..) But  , the problem is that we check data at the complet end of a job which may finish in a month or more . And if we resubmit , it will last another month or so . Which make project very hard to follow .  may be we need a way to check more often the raw data so to be able to reschedule earlier , only the bad subs , and not all of them ?  may be to have dirrectly acces to the rwa file each day so we will make our own project file ? 
legnolpi
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi  all  i m for option 1 , i prefer data than no data (as now in my locations , 4weeks cloudy time , i m highly depressed ..) But  , the problem is that we check data at the complet end of a job which may finish in a month or more . And if we resubmit , it will last another month or so . Which make project very hard to follow .  may be we need a way to check more often the raw data so to be able to reschedule earlier , only the bad subs , and not all of them ?  may be to have dirrectly acces to the rwa file each day so we will make our own project file ? 
Evening Option 1 is fine, the only issue with data access early is the data is stored in the interim locally on the PC and does not go onto a network share until it is finished, We can't let customers have direct PC access through the firewall system I am afraid as I am sure you will understand Remember this is only applicable while while the weather is changeable and for most of the year is not an issue Steve
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
yes , i understand , but may be changing the way to deliver the data . May be having some automatic rolling copy fo data from the shooting pc to the delivry NAS  then we will do ourselves the sorting and selecting ? 
legnolpi
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
[*]Data collected on main pier PC SSD drive[/*][*]Pete then checks using Blink[/*][*]Data is then zipped up[/*][*]Data is then transferred to our on site NAS drives as a hard backup copy[/*][*]Phil then uploads a copy of data to the server[/*][*]Data is then shared[/*][/list]Suggested system above lets call it - Option 2
  1. Data is taken partially on main pier PC SSD drive
    2. [/*]
  2. Pete checks partial data using Blink[/*]
  3. Partial data is zipped up[/*]
  4. Partial data is then transferred to on-site NAS[/*]
  5. Partial data is then uploaded to server[/*]
  6. Data is then shared [/*]
More data is then taken we repeat above steps
  1. Data is taken partially on main pier PC SSD drive
    2. [/*]
  2. Pete checks partial data using Blink[/*]
  3. Partial data is zipped up[/*]
  4. Partial data is then transferred to on-site NAS[/*]
  5. Partial data is then uploaded to server[/*]
  6. Data is then shared [/*]
More data is then taken we repeat above steps
  1. Data is taken partially on main pier PC SSD drive[/*]
  2. Pete checks partial data using Blink[/*]
  3. Partial data is zipped up[/*]
  4. Partial data is then transferred to on-site NAS[/*]
  5. Partial data is then uploaded to server[/*]
  6. Data is then share[/*]
To sum up Option 2 is very very labour intensive not to mention difficult to keep track of what data has been sent out of a data set already that could be taken over the course of say 10 nights of a 30 day period, We are a team of 5 people and honestly there are not hours in the day for use to keep checking small finished amounts of finished data then uploading it and keeping track of it. We could at times have to repeat this 5/10 times for one data set. I hope this explains my reticence because the logistics for us would be impossible for what is a wet season data blip that normally does not have a bearing upon the rest of the years data collection However, I will keep my thinking cap on to see if I can come up with a better solution Steve
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
I think Option 1 is the most practical for all concerned. The only frustration as has been mentioned is not knowing if you have good data until the end of a run. But then if you keep capturing, you never know which subs will be poor quality anyway. We have to realise that being part of a syndicate is not the same as owning a remote observatory. We can't 'look over the shoulder' of the Roboscopes piers while they are running. There are interesting approaches to checking the data as it is captured (for instance https://www.youtube.com/watch?v=a4IkAUZkXH0), but all that does is relieve us of a quality control check when we receive the data. I normally use blink to screen out any obviously defective subs, but I noticed with the M45 data that it was not until I had done the full WBPP on the subs that looked OK that the problems were visible. I probably need to include a sub-frame selector step in my process and be more data-driven instead of relying on visual inspection
old_eyes
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi all  i understand that option 2 is not workable , you cant blink each day data from 14piers .....    At home , if weather os ok for 2 days , i m overwhelm by data with one or 2 rig.... my own opinion is that as part of a syndicate , roboscope crew dont have to blink the data .  So you could send each day all the new data and we , syndicate user , have to select the data.    One good point of it is that we can process data more quickly and have some partial result that can be ok or even enought in other remote obsevatory , crew dont check the data as long that no error on equipement occurs during the night.  for paid per hour data , i t is a good think to be sure to deliver good data each time as we dont plan the observation.  florent
legnolpi
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi all  i understand that option 2 is not workable , you cant blink each day data from 14piers .....    At home , if weather os ok for 2 days , i m overwhelm by data with one or 2 rig.... my own opinion is that as part of a syndicate , roboscope crew dont have to blink the data .  So you could send each day all the new data and we , syndicate user , have to select the data.    One good point of it is that we can process data more quickly and have some partial result that can be ok or even enought in other remote obsevatory , crew dont check the data as long that no error on equipement occurs during the night.  for paid per hour data , i t is a good think to be sure to deliver good data each time as we dont plan the observation.  florent
Good morning Florent We have no problem with blinking or not blinking but obviously not blinking saves us a lot of time and will speed up things Piers-1 & 2 do not really suffer with dew Pier - 3 a small amount (This I think has now been sorted) Pier-5 a small amount for the first time ever (This I think has now been sorted) Pier-14 I think we have now sorted it (testing last nights data today) I have added an image below from Pier-10 this pier has been suffering quite badly for the last few weeks with the heavy humidity, howevert now the extra dew systems have finaly arrived in Spain and been added to the systems. This is B33 blue channel from last night with the dreaded Alnitak. I think this is acceptable data So to sum up if all members are happy for us not to blink and you all self check and re submit as needed, then we are happy to do this option as it will speed things up a hell of a lot on subscription piers? Steve 2021-12-14 11.43.29.jpg
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Morning all, I wondered if you had thought on the suggestion of us not blinking the data from our end and as it completes we simply zip it up and send it to members. This would make a massive difference in the speed you get the data as Phil can just zip it as with impunity as it sees the 100% complete icon Just let me know one way or another if you can please Steve
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Morning Steve,  Probably I'm misunderstanding this but I do not see any great benefit from the revised procedure you have described, and is not what I understood was being requested.  The following is my current understanding of how data captured nightly is being checked for us. After a night of data capture, time permitting, it is blinked by roboscopes to identify very obvious bad data. When this happens roboscopes (Peter) reduces the cumulative hours captured accordingly so that additional subs will automatically be taken. That is a tremendous benefit, saving us having to do so when we finally receive the data. This is most welcome and appreciated. This out of necessity can only be a quick check by Peter and still requires us to do a more thorough check and requires us to resubmit accordingly.  Before wasting more of your time by continuing this post, is my current understanding correct?  Cheers,  Ray   
NoMoreCloudyNights
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Ray I was responding to Florent's suggestion ? Cheers Steve Hi all  i understand that option 2 is not workable , you cant blink each day data from 14piers .....    At home , if weather os ok for 2 days , i m overwhelm by data with one or 2 rig.... my own opinion is that as part of a syndicate , roboscope crew dont have to blink the data .  So you could send each day all the new data and we , syndicate user , have to select the data.    One good point of it is that we can process data more quickly and have some partial result that can be ok or even enought in other remote obsevatory , crew dont check the data as long that no error on equipement occurs during the night.  for paid per hour data , i t is a good think to be sure to deliver good data each time as we dont plan the observation.  florent
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Good morning Steve,  I was responding likewise.   You can clear up my misunderstanding very easily. Florent, if I have it correct, is asking for all data collected nightly, unblinked, to be made available. That may well put pressure on another of your overworked staff members to download all syndicate data after any night data was collected.  You, if I have it correct and probably not, say data will only be made available, unblinked, upon 100% completion. That is a big difference. If I am understanding you correctly then we need Florent to confirm that you are delivering what he requested. I'll be happy to give my response then.  Cheers,  pita
NoMoreCloudyNights
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Good morning Steve,  I was responding likewise.   You can clear up my misunderstanding very easily. Florent, if I have it correct, is asking for all data collected nightly, unblinked, to be made available. That may well put pressure on another of your overworked staff members to download all syndicate data after any night data was collected.  You, if I have it correct and probably not, say data will only be made available, unblinked, upon 100% completion. That is a big difference. If I am understanding you correctly then we need Florent to confirm that you are delivering what he requested. I'll be happy to give my response then.  Cheers,  pita
That means I missed his full meaning also As I thought he was suggesting once data is finished rather than waiting for the data to be blinked by Pete (which can take 3/4/5 days) we simply zip and send to members This would speed things up for members obviously Steve
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Good morning Steve,  I was responding likewise.   You can clear up my misunderstanding very easily. Florent, if I have it correct, is asking for all data collected nightly, unblinked, to be made available. That may well put pressure on another of your overworked staff members to download all syndicate data after any night data was collected.  You, if I have it correct and probably not, say data will only be made available, unblinked, upon 100% completion. That is a big difference. If I am understanding you correctly then we need Florent to confirm that you are delivering what he requested. I'll be happy to give my response then.  Cheers,  pita
That means I missed his full meaning also As I thought he was suggesting once data is finished rather than waiting for the data to be blinked by Pete (which can take 3/4/5 days) we simply zip and send to members This would speed things up for members obviously Steve
Yes, if the data is not blinked until the collection is complete, then I think it is a waste of your valuable time to blink it. We are going to blink the data anyway, and we all have different thresholds for what we think is acceptable/good data. The current procedure is clearly not catching all the weak data (cf the M45 dataset - job 1141), so is it really worth your effort and the delays it introduces into the process? I am with Florent. Give us the 'warts and all' raw data and we can decide whether to resubmit or put in for additional data. old_eyes
old_eyes
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Honestly, it makes a huge difference and my preferred option if I am honest. far faster for all concerned We can trial on P1 & 14 for now if you wish ? Cheers steve
Asatron
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Steve,  Hopefully we are all on the same page as Florent, however, has something been overlooked?  Have a chat with Phil before committing to anything right now, and discuss how that will work.   I foresee a few issues which I'll gladly share, but not until you've had a word with the poor soul with more problems to deal with. If he is happy, then fair enough, but I believe it'll require more than a couple of daily butties to convince him.   Cheers,  Ray 
NoMoreCloudyNights
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi all  In my proposition , i ve also say that data transfert to the NAS should be automatic to spare time for roboscope crew .  if you have to do the transfer to NAS manualy, you cant do it each day, i understand.  it was just my opinion , you dont have to change all you workflow for me . As we are not behind the curtain , i can image there is some issue we dont see.  maybe a lot of increase traffic on you bandwith ? 
legnolpi
·
2 years ago
·
0 Likes
·
0 Votes
·
0 Comments
·
View Full Post