Process for Defining Standard Fields for MLSs

Oct 5, 2009  |  Michael Wurzer

In my last post, I mentioned that RETS data standards may get a boost from the COVE Group of MLSs starting to work together on further defining standard field names for their MLSs. Standard names have existed for a long time in RETS but they are rarely well populated. This new effort is to reinvigorate standard names and try to figure out how they can be changed so they are actually put into use.

Because there are other regional data standardization efforts across the country, I suggested in my last post that sharing the process for this work with the broader community would be helpful so that the work can be coordinated with work other groups are doing. One of the most basic first steps is defining a format for publishing the agreed upon fields. Spreadsheets tend to be the easiest way for the broadest group of people to access the results, and so they are most commonly used in data standardization efforts.

One of the challenges with spreadsheets, though, is that they are two dimensional and the listing data often is nested, especially when it comes to fields with a list of values. How do you best show the field name and the list of values? One approach is to combine them into one sheet and another is to include the list of values on a separate sheet as shown in the example below (though I couldn’t figure out how to link the two sheets together in Google documents).

Another question is whether to include the data type in this initial effort. In the past, defining data type has created a lot of consternation where, for example, some MLSs use a list of values for number of bedrooms while most others use a numeric value. If this effort is to have meaning and usefulness long term, I think we have to define data types as well as names. Lastly, even though RETS schema doesn’t define fields by property type, I suggest the spreadsheet contain a property type column to define those fields particular to certain property types.

I think using Google Documents or some other web based spreadsheet would make sharing and publishing the information on sites like RETS.org much easier for everyone. Sharing is critical for getting input and feedback, and building consensus around the results. For example, if you think of a better idea for how to format the spreadsheet, you actually can edit the spreadsheet above and the changes will be saved. That’s pretty cool. Google Spreadsheets also can grant only specific people permission for editing.

So, does this approach work for groups to create and share standard fields in an effort to come up with widely adopted standard fields? One of the core objectives is to get as many fields commonly searchable across all MLSs as possible. Is that 50 fields? 100? More? If there aren’t many current fields, will MLSs be willing to change their current fields to adopt to new standards? Lots of questions here but I know there are many MLSs grappling with these exact problems right now and so this is the time to put keyboard to spreadsheet and define as many standard fields as possible.

7 Responses to “Process for Defining Standard Fields for MLSs”

  1. Marc Smith says:

    Mike,

    I raised this point with Paul Stuziak the night before this was brought up on the technical track. It originated from work we were doing with Rosetta. To cut a long story short the discussion was my take on the first phase towards normalising data that had the least development impact or rather was the least effort for a Vendor or MLS to implement.

    As standard names already exist as a mechanism it should require no development and should for the majority of systems just be configuration work.

    As such my view is that the ‘first’ phase should only require standard names to be completed and not include any requirements vis a vis data types or enumerations, although recommendations at this stage would not be a bad idea.

    The current spreadsheet on the rets.org site, which I haven’t looked at in while, isn’t at all clear if memory serves me right, so I think your idea of generating a new one is great.

    My thoughts for the initial iteration of the structure should be:

    1) Resource Standard Name,
    2) Class Standard Name*,
    3) Table Standard Name,
    4) Description,
    5) Recommended Data Type

    * This can have an entry that is CommonTableFields which is not a valid class standard name value but denotes that these standard names are valid for all classes under this resource.

    We’ve also got a utility that pulls metadata from any set of systems, extracts the standard names and generates a csv file with the collated results of usage of standard names across all the systems.

    If you can organise people to send me credentials and login urls to as many systems we can probably generate an initial document that takes into account current usage across all the systems that participate. Which might be a good starting point for a document.

    If you want to drop me an e-mail, I can send you an anonymised version of this file that includes a sample of 9 MLSs.

    Cheers,
    Marc

  2. We just recently added Residential Rentals to our FlexMLS system. We started with a clone of our regular Residential database and then we added some rental specific fields from a neighboring MLS.

    The problem is that we were told that the other MLS has some of these fields defined in specific columns which we are already using for other data. If we didn’t use the exact field and name then we could not use data sharing.

    One of the primary rules of relational database design is that it’s not supposed to matter where a column is located in a table. The name of the column shouldn’t matter either.

    One way to deal with this issue is to use mapping tables. It adds another layer of complexity but it works.

    Malcolm

  3. mwurzer says:

    Malcolm, you can share data with non-common fields but searching across the same main list field requires that the data be stored in the same field. You’re right that we could map the fields and search across multiple columns simultaneously for the single data value, but that’s not efficient from a database perspective and so we currently require that the same columns be used for searching. We do not require that the field descriptions be the same, which you can see by looking at the variety of names in the system for price, bedrooms, bathrooms and other fields. What we do require, however, is that we have to know that the fields are actually the same so we can map them. We know that for the core address fields, price, bedrooms, dates, etc., but there are many other custom fields which, without knowing they are the same, we are left with requiring that the description be the same. As long as we know, however, we can map them as we’re doing with the standard fields. This is exactly why data standards are so important, so we can expand the common fields and make data sharing work more comprehensively out of the box without field mapping.

  4. […] typically discuss data standards in relation to MLSs wanting to share data or regionalize, but this is a good example […]

  5. I figured there was probably a performance issue involved with the column mapping but I understood that the descriptions had to match as well.

    We have some UserDefinedXX fields that are pretty important to us. How do we know what is the “best” use for them among our many neighbors and how would we begin to get them aligned without disrupting the existing systems?

    I’m willing to work on this as the MLS Committee Chair Elect but I believe it will require a lot of work from FBS. There are a lot of common data fields but also some that are unique by area. Does FlexMLS have enough configurable fields to accommodate them all?

    Years ago and before adoption of FlexMLS (a fabulous system in my opinion), I understand our MLS made many changes to adapt to a neighbor. After all that it wound up failing so I don’t see our members having any taste for compromise that would require us to giving up functionality.

    Malcolm
    (who hates making table changes to production systems) 😉

  6. mwurzer says:

    Yes, we have plenty of fields; an unlimited number, in fact, with detail fields.

    You’re also right that it will require a lot of work from FBS, which is why we’ve been working so hard to try to organize the efforts on as big of a scale as possible. Moving toward one standard is so much easier than doing it over and over and over again for each MLS and region.

    As described in earlier posts, there is a newly reinvigorated effort by the COVE Group to work with RESO to define standard fields. That effort is going to start simple, with some spreadsheets defining the fields.

    In your area, we’ve been discussing this already with Scranton and several other MLSs and we recommend the group getting together to define and publish the common fields you want for your region. We’ll then take that effort and coordinate it with the RESO effort.

    I’m glad you’re willing to jump in here, it isn’t that daunting of a task but just requires some detailed work to get it done.

  7. […] where standards could be of great benefit.  Creating standards, however, is very, very difficult, especially at a national level.  Of course, one way to create a national standard is to create one MLS.  The problem with that […]