Palo alto index field error

hxshira

Hello,

We are experiencing an issue with the Palo Alto logs and default dashboards.

After importing the built-in dashboards, some panels display errors related to the field bytes.total, with the message:

Field "bytes.total" is invalid for use with the "Sum" aggregation.

I attempted to change the field type of bytes.total to make it compatible with the aggregation, but was unable to do so successfully.

Could you please assist in resolving this issue?

SzymonC

Sure!

Most likely this is due a mapping error. Mapping is section of the index and it describes what type field is (string, integer, float, boolean, goe, etc.).
To make sure that's the case, I'd like you to run GET command through UI module called Dev Tools. I don't know what's your index name, so I used stream-netflow, which would have similiar field. Try to extract it and send it over.
It looks like this:
GET stream-netflow-*/_mapping/field/netflow.bytes

My response is like below. As you can see, the type of that field is long, which should work in my case. But the question is what type of field you have.

hxshira

Hi,

As you can see, the field type is a keyword. How can I change this so that the visualizations works correctly?

SzymonC

Exactly - keyword is form of string, so that's the reason why mathematical operation on visualizations ends up with errors.
Mapping is form when index is created. In your case new palo index is created every day. When database is creating index it's doing it according to one or multiple templates. Template itself is an object inside the database and is used exactly for that - to provide instructions on how indices should be created.
So to answer your question: you need to update your template in the mapping section. Here's a different topic, where I'm mentioning templates: https://community.energylogserver.com/d/67-unknown-fields/2
I'm not sure what's your template name, but you can find correct one by looking at the 1st element - index_patterns. It will match the name of the index, so most like you're looking for something like this:

  "index_patterns": [
    "*syslog-net-fw-palo*"
  ],

Or it can have something general, like *palo*. In it you'll need to update the type of the field to correct value, most likely byte. It'll be in the mapping section and should look something like this:

"bytes": {
  "total": {
    "type": "byte"
  }
}

Always remember of doing backup of your existing, working template before any change. You can copy the content and save it in some txt file.
Assuming that you'll find correct template and field and correct it, next day you can query the index API like before and you should see correct value. From now on your visualizations should work. Just note that if you'll be searching last 24 hours, visualization will still encounter error, because it'll try to visualize data from index with bad mapping.

If you're not sure about how to approach it and you're not feeling confident on doing it on your own, please reach out to our support in order to get professional services assistance on that.

Hope that helps!