Kryo ClassNotFoundException внезапно брошен

Прочитав много постов, я не нашел ответа на свою проблему с kryo.

Мы стараемся реализовать Kryo (v4.0.0) в нашей системе, которые сериализуют/десериализуют очень большое количество сообщений между двумя серверами.

Наша проблема заключается в том, что сериализация / десериализация может работать нормально в течение нескольких минут (поэтому более тысячи сообщений) и внезапно перестать работать со следующим исключением :

2016-11-09 13:23:32,968 [AMQP_consumer_blk-ic-p.tradair.com_TMSOutWL_TMSOutWL_streaming-1] ERROR Driver handleDelivery -
com.esotericsoftware.kryo.KryoException: Unable to find class: ^AA^]^A^@^A^B^C^_^C ^C!^C^A59927910321794
Serialization trace:
payload (com.trade.common.messagebus.AMQPPacket)
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813) ~[kryo-shaded-4.0.0.jar:?]
        at com.tradair.common.serialization.kryo.KryoSerializer.deserialize(KryoSerializer.java:60) ~[common-4.7.3.12.jar:?]
        at com.trade.common.messagebus.AbstractMessageBusConsumer

After reading many posts, i did not find an answer to my issue with kryo.

We are trying to implement Kryo (v4.0.0) in our system that serialize/deserialize a very big quantity of messages between two servers.

Our issue is that the serialization/deserialization can work fine during several minutes (so more than thousand messages) and suddenly stop to work with the following exception :

2016-11-09 13:23:32,968 [AMQP_consumer_blk-ic-p.tradair.com_TMSOutWL_TMSOutWL_streaming-1] ERROR Driver handleDelivery -
com.esotericsoftware.kryo.KryoException: Unable to find class: ^AA^]^A^@^A^B^C^_^C ^C!^C^A59927910321794
Serialization trace:
payload (com.trade.common.messagebus.AMQPPacket)
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:118) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:540) ~[kryo-shaded-4.0.0.jar:?]
        at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813) ~[kryo-shaded-4.0.0.jar:?]
        at com.tradair.common.serialization.kryo.KryoSerializer.deserialize(KryoSerializer.java:60) ~[common-4.7.3.12.jar:?]
        at com.trade.common.messagebus.AbstractMessageBusConsumer$1.handleDelivery(AbstractMessageBusConsumer.java:106) ~[common-4.7.3.12.jar:?]
        at com.rabbitmq.client.impl.ConsumerDispatcher$5.run(ConsumerDispatcher.java:144) ~[amqp-client-3.4.3.jar:?]
        at com.rabbitmq.client.impl.ConsumerWorkService$WorkPoolRunnable.run(ConsumerWorkService.java:95) ~[amqp-client-3.4.3.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_40]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_40]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
Caused by: java.lang.ClassNotFoundException: ^AA^]^A^@^A^B^C^_^C ^C!^C^A59927910321794
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_40]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_40]
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_40]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_40]
        at java.lang.Class.forName0(Native Method) ~[?:1.8.0_40]
        at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_40]
        at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154) ~[kryo-shaded-4.0.0.jar:?]
        ... 12 more

You can see that Kryo does not succeed to read the class name (Unable to find class: ^AA^]^A^@^A^B^C^_^C ^C!^C^A59927910321794).

My serialize class :

public class KryoSerializer extends AbstractSerializer {

    private Logger logger = LoggerFactory.getLogger(KryoSerializer.class);
    protected Kryo kryo;
    protected Output output;
    protected Input input;
    protected ByteArrayOutputStream bos;
    protected ByteArrayInputStream bis;

    public KryoSerializer(List<Class> list) {
        kryo = new Kryo();
        kryo.setRegistrationRequired(false);
        kryo.setReferences(true);
        for (Class cl : list) {
            kryo.register(cl);
        }
    }

    @Override
    public <T> byte[] serialize(T msg) {
        try {
            bos = new ByteArrayOutputStream();
            output = new Output(bos);
            kryo.writeClassAndObject(output, msg);
            output.flush();
            return bos.toByteArray();
        } finally {
            try {
                if (output != null) {
                    output.close();
                }
                if (bos != null) {
                    bos.close();
                }
            } catch (IOException eio) {
                logger.warn("", eio);
            }
        }
    }

    @Override
    public <T> T deserialize(byte[] bytesArray) {
        try {
            bis = new ByteArrayInputStream(bytesArray);
            input = new Input(bis);
            return (T) kryo.readClassAndObject(input);
        } finally {
            try {
                if (input != null) {
                    input.close();
                }
                if (bis != null) {
                    bis.close();
                }
            } catch (IOException eio) {
                logger.warn("", eio);
            }
        }
    }
}

.handleDelivery(AbstractMessageBusConsumer.java:106) ~[common-4.7.3.12.jar:?]
at com.rabbitmq.client.impl.ConsumerDispatcher.run(ConsumerDispatcher.java:144) ~[amqp-client-3.4.3.jar:?]
at com.rabbitmq.client.impl.ConsumerWorkService$WorkPoolRunnable.run(ConsumerWorkService.java:95) ~[amqp-client-3.4.3.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_40]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_40]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_40]
Caused by: java.lang.ClassNotFoundException: ^AA^]^A^@^A^B^C^_^C ^C!^C^A59927910321794
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_40]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_40]
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) ~[?:1.8.0_40]
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_40]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_40]
at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_40]
at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154) ~[kryo-shaded-4.0.0.jar:?]
... 12 more

Вы можете видеть, что Kryo не удается прочитать имя класса (не удается найти класс: ^AA^]^A^@^A^B^C^_^C ^C!^C^A59927910321794).

Мой класс сериализации :

public class KryoSerializer extends AbstractSerializer {

    private Logger logger = LoggerFactory.getLogger(KryoSerializer.class);
    protected Kryo kryo;
    protected Output output;
    protected Input input;
    protected ByteArrayOutputStream bos;
    protected ByteArrayInputStream bis;

    public KryoSerializer(List<Class> list) {
        kryo = new Kryo();
        kryo.setRegistrationRequired(false);
        kryo.setReferences(true);
        for (Class cl : list) {
            kryo.register(cl);
        }
    }

    @Override
    public <T> byte[] serialize(T msg) {
        try {
            bos = new ByteArrayOutputStream();
            output = new Output(bos);
            kryo.writeClassAndObject(output, msg);
            output.flush();
            return bos.toByteArray();
        } finally {
            try {
                if (output != null) {
                    output.close();
                }
                if (bos != null) {
                    bos.close();
                }
            } catch (IOException eio) {
                logger.warn("", eio);
            }
        }
    }

    @Override
    public <T> T deserialize(byte[] bytesArray) {
        try {
            bis = new ByteArrayInputStream(bytesArray);
            input = new Input(bis);
            return (T) kryo.readClassAndObject(input);
        } finally {
            try {
                if (input != null) {
                    input.close();
                }
                if (bis != null) {
                    bis.close();
                }
            } catch (IOException eio) {
                logger.warn("", eio);
            }
        }
    }
}

1 ответ

  1. Как я знаю, класс Kryo не является потокобезопасным. Если вы используете его в параллельной среде, каждый раз необходимо создавать новый экземпляр. Вроде этого:

    bos = new ByteArrayOutputStream();
    output = new Output(bos);
    new Kryo().writeClassAndObject(output, msg);
    output.flush();
    return bos.toByteArray();
    

    Я использовал, чтобы решить похожую проблему таким образом.